Sep 10 21:07:57.821223 ip-10-0-131-178 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 21:07:57.821234 ip-10-0-131-178 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 21:07:57.821240 ip-10-0-131-178 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 21:07:57.821443 ip-10-0-131-178 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 21:08:08.070775 ip-10-0-131-178 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 21:08:08.070797 ip-10-0-131-178 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7fe921e22bb04c4ca783e510305b21ac --
Sep 10 21:09:41.994592 ip-10-0-131-178 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 21:09:42.416403 ip-10-0-131-178 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:09:42.416403 ip-10-0-131-178 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 21:09:42.416403 ip-10-0-131-178 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:09:42.416403 ip-10-0-131-178 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 21:09:42.416403 ip-10-0-131-178 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:09:42.419579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.419477    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 21:09:42.424593 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424578    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.424593 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424594    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424598    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424601    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424604    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424608    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424611    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424613    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424622    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424625    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424628    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424630    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424633    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424636    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424638    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424641    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424643    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424646    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424650    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424653    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424656    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.424659 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424658    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424661    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424664    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424667    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424670    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424672    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424675    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424678    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424680    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424683    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424686    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424689    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424692    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424694    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424697    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424699    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424702    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424704    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424707    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424710    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.425125 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424714    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424717    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424720    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424723    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424725    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424728    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424730    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424733    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424735    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424738    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424740    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424743    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424745    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424747    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424751    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424754    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424758    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424761    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424765    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.425631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424768    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424771    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424773    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424776    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424778    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424781    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424783    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424786    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424788    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424791    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424794    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424796    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424799    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424801    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424804    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424806    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424809    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424811    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424814    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424817    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.426086 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424820    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424822    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424825    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424827    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424830    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424833    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424836    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424839    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424842    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424844    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424847    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424849    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424852    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424855    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424858    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424860    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424863    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424865    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.426579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.424868    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.427307 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427294    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.427307 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427306    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427310    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427314    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427318    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427321    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427325    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427328    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427331    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427334    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427336    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427339    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427342    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427344    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427347    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427350    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427352    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427355    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427357    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427360    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427362    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.427366 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427365    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427368    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427370    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427373    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427376    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427379    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427381    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427384    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427386    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427389    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427392    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427394    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427396    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427399    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427402    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427404    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427407    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427409    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427412    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427415    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.427858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427417    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427420    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427422    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427425    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427427    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427431    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427434    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427436    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427439    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427441    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427444    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427447    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427450    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427453    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427456    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427458    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427461    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427463    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427466    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.428357 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427469    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427471    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427474    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427476    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427479    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427481    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427484    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427488    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427491    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427494    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427497    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427499    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427503    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427505    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427508    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427511    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427514    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427516    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427519    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.428822 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427522    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427524    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427526    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427529    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427532    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427535    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427537    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427539    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427542    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427545    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427547    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427551    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427554    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427557    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427560    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427563    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427566    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427568    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427571    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427573    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.429297 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427918    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427923    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427926    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427928    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427931    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427934    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427937    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427940    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427942    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427945    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427947    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427950    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427952    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427955    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427958    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427961    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427963    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427966    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427968    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427971    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.429791 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427974    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427977    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427979    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427983    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427987    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427990    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427993    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427996    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.427998    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428001    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428004    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428006    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428009    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428012    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428014    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428017    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428020    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428023    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428025    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428028    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.430308 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428031    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428033    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428036    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428042    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428045    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428048    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428051    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428054    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428056    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428059    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428062    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428065    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428067    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428070    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428073    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428075    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428078    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428080    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428083    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.430828 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428085    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428088    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428090    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428093    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428095    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428098    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428100    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428103    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428105    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428108    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428110    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428113    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428116    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428118    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428121    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428124    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428128    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428131    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428133    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428136    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.431365 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428138    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428142    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428145    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428148    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428152    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428155    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428158    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428161    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428164    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428167    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428170    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428172    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428175    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428178    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428180    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428182    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428185    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428188    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428190    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428193    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.431984 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428265    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428316    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428323    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428327    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428331    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428335    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428339    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428343    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428348    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428351    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428355    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428358    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428362    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428365    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428368    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428371    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428374    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428377    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428380    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428383    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428386    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428389    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428393    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428396    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 21:09:42.432595 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428399    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428402    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428406    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428409    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428412    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428415    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428418    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428421    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428424    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428427    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428430    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428433    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428436    2580 flags.go:64] FLAG: --enable-server="true"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428439    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428444    2580 flags.go:64] FLAG: --event-burst="100"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428447    2580 flags.go:64] FLAG: --event-qps="50"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428451    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428454    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428456    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428460    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428463    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428466    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428469    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428471    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428474    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 21:09:42.433252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428477    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428480    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428482    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428486    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428488    2580 flags.go:64] FLAG: --feature-gates=""
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428492    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428495    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428498    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428501    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428503    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428506    2580 flags.go:64] FLAG: --help="false"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428509    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428512    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428515    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428517    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428521    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428524    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428527    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428530    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428532    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428535    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428538    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428541    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428544    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 10 21:09:42.433954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428547    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428550    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428553    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428556    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428559    2580 flags.go:64] FLAG: --lock-file=""
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428561    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428564    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428567    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428572    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428575    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428578    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428581    2580 flags.go:64] FLAG: --logging-format="text"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428584    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428587    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428590    2580 flags.go:64] FLAG: --manifest-url=""
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428593    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428598    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428601    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428605    2580 flags.go:64] FLAG: --max-pods="110"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428608    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428611    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428613    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428616    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428621    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428624    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 21:09:42.434619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428627    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428634    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428637    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428640    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428643    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428646    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428651    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428654    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428657    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428660    2580 flags.go:64] FLAG: --port="10250"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428663    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428666    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ec9b97d10100a371"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428669    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428672    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428675    2580 flags.go:64] FLAG: --register-node="true"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428677    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428681    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428684    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428687    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428690    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428693    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428696    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428699    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428702    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428704    2580 flags.go:64] FLAG: --runonce="false"
Sep 10 21:09:42.435285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428708    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428711    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428713    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428716    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428719    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428723    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428726    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428729    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428731    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428734    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428737    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428740    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428743    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428746    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428749    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428754    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428757    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428760    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428763    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428766    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428769    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428771    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428774    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428777    2580 flags.go:64] FLAG: --v="2"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428781    2580 flags.go:64] FLAG: --version="false"
Sep 10 21:09:42.435907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428784    2580 flags.go:64] FLAG: --vmodule=""
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428788    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.428791    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428904    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428908    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428911    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428913    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428916    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428919    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428921    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428924    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428926    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428930    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428933    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428936    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428938    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428940    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428943    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428945    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428948    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.436858 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428952    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428955    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428958    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428960    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428963    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428965    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428968    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428970    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428973    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428975    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428978    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428980    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428983    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428985    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428988    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428990    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428993    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428995    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.428998    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429001    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.437575 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429003    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429006    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429008    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429011    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429014    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429017    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429019    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429022    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429024    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429027    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429029    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429032    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429036    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429038    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429041    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429044    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429047    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429049    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429052    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429055    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.438078 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429057    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429060    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429063    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429066    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429070    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429074    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429078    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429081    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429083    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429087    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429091    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429094    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429097    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429100    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429103    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429106    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429110    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429113    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429116    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.438579 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429118    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429122    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429125    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429128    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429131    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429136    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429139    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429142    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429146    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429150    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429152    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429155    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429158    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429160    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429163    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429165    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429168    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429170    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429173    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.439046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429175    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429193    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429196    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.429200    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.429842    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.438266    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.438291    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.438979    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.438986    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439045    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439051    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439055    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439058    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439061    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439063    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.439529 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439066    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439069    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439072    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439075    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439078    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439080    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439083    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439086    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439089    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439091    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439095    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439099    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439102    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439105    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439108    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439111    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439114    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439117    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439120    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439123    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.439898 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439126    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439129    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439132    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439135    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439138    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439140    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439143    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439146    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439149    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439151    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439154    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439157    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439159    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439162    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439164    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439167    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439169    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439172    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439174    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.440462 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439177    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439181    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439184    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439187    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439189    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439192    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439195    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439197    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439200    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439204    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439207    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439211    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439213    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439216    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439219    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439221    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439224    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439227    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439230    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439232    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.440911 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439235    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439238    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439241    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439243    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439246    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439249    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439252    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439256    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439259    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439261    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439264    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439267    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439279    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439282    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439285    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439288    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439291    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439294    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439296    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.441429 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439299    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439301    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439304    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439306    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439309    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439312    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439314    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439317    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439319    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439322    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439325    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439328    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439330    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439333    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439336    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.441896 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.439341    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439453    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439458    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439461    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439464    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439467    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439469    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439472    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439474    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439477    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439480    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439483    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439485    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439489    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439492    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439494    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439497    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439500    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439502    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439505    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439507    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:09:42.442265 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439510    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439512    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439515    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439518    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439520    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439523    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439526    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439529    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439531    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439535    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439540    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439543    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439546    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439549    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439551    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439554    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439556    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439559    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439562    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:09:42.442891 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439565    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439567    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439570    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439572    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439575    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439577    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439581    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439584    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439586    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439590    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439594    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439597    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439600    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439603    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439606    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439609    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439612    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439614    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439617    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:09:42.443362 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439620    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439623    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439626    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439628    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439631    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439634    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439637    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439639    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439642    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439645    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439648    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439651    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439654    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439657    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439661    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439664    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439667    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439669    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439672    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:09:42.443812 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439674    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439678    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439680    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439683    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439685    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439688    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439690    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439693    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439695    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439698    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439700    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439703    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439706    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439708    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439711    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439713    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439716    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439718    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439721    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439723    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:09:42.444284 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439726    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:42.439728    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.439733    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.439826    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.442884    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.443821    2580 server.go:1019] "Starting client certificate rotation"
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.443916    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:09:42.444766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.443954    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:09:42.446481 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.446468    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:09:42.447558 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.447539    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:09:42.456735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.456636    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 10 21:09:42.460027 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.460014    2580 log.go:25] "Validated CRI v1 image API"
Sep 10 21:09:42.460890 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.460871    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 21:09:42.466809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.466789    2580 fs.go:135] Filesystem UUIDs: map[757df626-14ac-4665-a9ba-8dce78401481:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8f0a151e-de36-41c1-bc7d-e723d3b2475d:/dev/nvme0n1p3]
Sep 10 21:09:42.466903 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.466808    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 21:09:42.473718 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.473603    2580 manager.go:216] Machine: {Timestamp:2026-09-10 21:09:42.47240223 +0000 UTC m=+0.351111283 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098354 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e34331e86d38dbab7bf22b80e7575 SystemUUID:ec2e3433-1e86-d38d-bab7-bf22b80e7575 BootID:7fe921e2-2bb0-4c4c-a783-e510305b21ac Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:0b:2d:9a:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:0b:2d:9a:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:6c:31:3d:d8:bd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 21:09:42.473718 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.473705    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 21:09:42.473870 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.473800    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 21:09:42.476564 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.476548    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:09:42.477525 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.477497    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 21:09:42.477647 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.477529    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 21:09:42.478733 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.478721    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 21:09:42.478784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.478735    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 21:09:42.478784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.478743    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:09:42.479772 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.479763    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:09:42.480618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.480608    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 21:09:42.483500 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483490    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:09:42.483632 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483623    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 21:09:42.483662 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483651    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 21:09:42.483698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483666    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 21:09:42.483698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483683    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 21:09:42.483698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483691    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 10 21:09:42.483814 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.483699    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 21:09:42.484866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.484846    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:09:42.484972 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.484875    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:09:42.487800 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.487780    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 21:09:42.489552 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.489539    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 21:09:42.489611 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.489569    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 21:09:42.492084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492069    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492088    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492094    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492099    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492107    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492112    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492125    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492130    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492137    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 21:09:42.492140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492143    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 21:09:42.492403 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492151    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 21:09:42.492403 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492156    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 21:09:42.492873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492863    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 21:09:42.492873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.492872    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 21:09:42.495464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.495443    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 21:09:42.495528 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.495485    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 21:09:42.495528 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.495494    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 21:09:42.496815 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.496802    2580 server.go:1268] "Started kubelet"
Sep 10 21:09:42.496924 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.496882    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 21:09:42.496978 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.496942    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 21:09:42.496978 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.496904    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 21:09:42.497630 ip-10-0-131-178 systemd[1]: Started Kubernetes Kubelet.
Sep 10 21:09:42.498523 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.498505    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 21:09:42.499616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.499601    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 21:09:42.501146 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.501123    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 21:09:42.502966 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.502952    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 21:09:42.503606 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.503590    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 21:09:42.503813 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.503793    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.505212 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505183    2580 factory.go:55] Registering systemd factory
Sep 10 21:09:42.505212 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505202    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 10 21:09:42.505508 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505493    2580 factory.go:153] Registering CRI-O factory
Sep 10 21:09:42.505592 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505583    2580 factory.go:223] Registration of the crio container factory successfully
Sep 10 21:09:42.505701 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505679    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 21:09:42.505701 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505686    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505715    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505743    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505764    2580 factory.go:103] Registering Raw factory
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505779    2580 manager.go:1196] Started watching for new ooms in manager
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505790    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 21:09:42.505853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.505798    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 21:09:42.506224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.506204    2580 manager.go:349] Starting recovery of all containers
Sep 10 21:09:42.506980 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.504876    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-178.ec2.internal.18d411da966696de  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-178.ec2.internal,UID:ip-10-0-131-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-178.ec2.internal,},FirstTimestamp:2026-09-10 21:09:42.49677795 +0000 UTC m=+0.375487002,LastTimestamp:2026-09-10 21:09:42.49677795 +0000 UTC m=+0.375487002,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-178.ec2.internal,}"
Sep 10 21:09:42.512912 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.512880    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 21:09:42.513037 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.513007    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 21:09:42.518285 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.518261    2580 manager.go:354] Recovery completed
Sep 10 21:09:42.522383 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.522370    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.523969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.523945    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.524048 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.523976    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.524048 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.523989    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.524652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.524637    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 21:09:42.524725 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.524652    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 21:09:42.524725 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.524669    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:09:42.524725 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.524642    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.525242 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.525222    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xv4lh"
Sep 10 21:09:42.525427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.525413    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.525475 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.525443    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.525475 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.525457    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.525693 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.525639    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-178.ec2.internal.18d411da98057951  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-178.ec2.internal,UID:ip-10-0-131-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-178.ec2.internal,},FirstTimestamp:2026-09-10 21:09:42.523967825 +0000 UTC m=+0.402676878,LastTimestamp:2026-09-10 21:09:42.523967825 +0000 UTC m=+0.402676878,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-178.ec2.internal,}"
Sep 10 21:09:42.526043 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.526030    2580 policy_none.go:49] "None policy: Start"
Sep 10 21:09:42.526074 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.526047    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 21:09:42.526074 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.526057    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 21:09:42.527181 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.527167    2580 policy_none.go:47] "Start"
Sep 10 21:09:42.531187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.531168    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xv4lh"
Sep 10 21:09:42.537129 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.537064    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-178.ec2.internal.18d411da9805b10d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-178.ec2.internal,UID:ip-10-0-131-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-178.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-178.ec2.internal,},FirstTimestamp:2026-09-10 21:09:42.523982093 +0000 UTC m=+0.402691151,LastTimestamp:2026-09-10 21:09:42.523982093 +0000 UTC m=+0.402691151,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-178.ec2.internal,}"
Sep 10 21:09:42.551820 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.551806    2580 manager.go:337] "Starting Device Plugin manager"
Sep 10 21:09:42.551903 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.551840    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 21:09:42.551903 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.551852    2580 server.go:87] "Starting device plugin registration server"
Sep 10 21:09:42.552132 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.552119    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 21:09:42.552201 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.552133    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 21:09:42.552654 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.552368    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 21:09:42.552654 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.552455    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 21:09:42.552654 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.552464    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 21:09:42.553127 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.553086    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 21:09:42.553199 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.553139    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.603454 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.603433    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 21:09:42.604478 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.604461    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 21:09:42.604570 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.604506    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 21:09:42.604570 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.604526    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 21:09:42.604570 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.604565    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 21:09:42.607022 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.607006    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:42.653086 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.653062    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.653887 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.653869    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.653969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.653896    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.653969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.653907    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.653969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.653927    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.661749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.661735    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.661797 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.661756    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-178.ec2.internal\": node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.676520 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.676452    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.704806 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.704783    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"]
Sep 10 21:09:42.704871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.704849    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.706203 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.706186    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.706335 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.706210    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.706335 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.706219    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.707433 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.707421    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.707609 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.707593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.707647 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.707626    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.708158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708138    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.708158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708154    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.708333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708165    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.708333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708177    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.708333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708191    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.708333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.708194    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.709417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.709401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.709496 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.709428    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:09:42.710072 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.710055    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:09:42.710143 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.710079    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:09:42.710143 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.710088    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:09:42.724799 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.724782    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-178.ec2.internal\" not found" node="ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.728754 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.728738    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-178.ec2.internal\" not found" node="ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.776829 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.776808    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.806241 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.806212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.806315 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.806251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.877279 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.877248    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:42.906613 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.906593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.906661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.906629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.906661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.906653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52c9ae203a047aa285585b14707a01ad-config\") pod \"kube-apiserver-proxy-ip-10-0-131-178.ec2.internal\" (UID: \"52c9ae203a047aa285585b14707a01ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.906747 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.906673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.906747 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:42.906703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf54620341eeda8eba6e62b7a0c0d35c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal\" (UID: \"cf54620341eeda8eba6e62b7a0c0d35c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:42.977887 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:42.977836    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:43.007357 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.007339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52c9ae203a047aa285585b14707a01ad-config\") pod \"kube-apiserver-proxy-ip-10-0-131-178.ec2.internal\" (UID: \"52c9ae203a047aa285585b14707a01ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.007408 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.007375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52c9ae203a047aa285585b14707a01ad-config\") pod \"kube-apiserver-proxy-ip-10-0-131-178.ec2.internal\" (UID: \"52c9ae203a047aa285585b14707a01ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.027486 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.027469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.032255 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.032240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.078172 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.078152    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:43.178668 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.178642    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:43.279235 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.279158    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:43.382604 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.382568    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-178.ec2.internal\" not found"
Sep 10 21:09:43.393638 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.393617    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:43.444093 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.444069    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 21:09:43.444566 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.444218    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:09:43.444566 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.444226    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:09:43.459876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.459858    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:43.483948 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.483927    2580 apiserver.go:52] "Watching apiserver"
Sep 10 21:09:43.495768 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.495747    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 21:09:43.496116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.496095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-kjzvc","openshift-cluster-node-tuning-operator/tuned-8s8kk","openshift-dns/node-resolver-5zqd4","openshift-multus/multus-additional-cni-plugins-6zwbc","openshift-multus/multus-cbngr","openshift-network-diagnostics/network-check-target-4rf6f","openshift-network-operator/iptables-alerter-sd2hh","openshift-ovn-kubernetes/ovnkube-node-dshvv","kube-system/konnectivity-agent-xl8x9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f","openshift-image-registry/node-ca-fg6nr","openshift-multus/network-metrics-daemon-p2vbb"]
Sep 10 21:09:43.497709 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.497693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.497809 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.497756    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:43.499097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.499080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.500667 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.500648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.501343 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.501317    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dwwnr\""
Sep 10 21:09:43.501442 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.501321    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.501523 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.501504    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.502066 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.502049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.502996 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.502977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.503076 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.503026    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 21:09:43.503263 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.503245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.503327 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.503304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.503895 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.503875    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.504108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.503931    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gtdsc\""
Sep 10 21:09:43.504108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.504009    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 21:09:43.504299 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.504209    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 21:09:43.504299 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.504212    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.504635 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.504619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:43.504752 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.504691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:43.504948 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.504933    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.505244 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.505228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx2q5\""
Sep 10 21:09:43.505350 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.505281    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 21:09:43.505350 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.505328    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-khssb\""
Sep 10 21:09:43.505517 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.505505    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 21:09:43.506342 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.506327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.508167 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.508132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.508167 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.508165    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.508644 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.508627    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 21:09:43.509040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vfdnx\""
Sep 10 21:09:43.509083 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.509670 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-host\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.509786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.509786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.509786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-bin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-conf\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-sys\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r9db\" (UniqueName: \"kubernetes.io/projected/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-kube-api-access-7r9db\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f5e3ce5-4e7a-416d-876d-88f5928a3755-host-slash\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.509947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-tuned\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.509986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-system-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-socket-dir-parent\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-k8s-cni-cncf-io\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-kubelet\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-etc-kubernetes\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qvgz\" (UniqueName: \"kubernetes.io/projected/5f5e3ce5-4e7a-416d-876d-88f5928a3755-kube-api-access-8qvgz\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-kubernetes\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-lib-modules\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-tmp-dir\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cnibin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-kubelet-config\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-dbus\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dwjz\" (UniqueName: \"kubernetes.io/projected/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-kube-api-access-9dwjz\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-netns\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-multus-certs\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-tmp\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysconfig\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-systemd\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-run\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-var-lib-kubelet\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-hosts-file\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cnibin\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ng9v\" (UniqueName: \"kubernetes.io/projected/76912b7e-9cdd-4879-a340-e66ca7c0c79c-kube-api-access-2ng9v\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-multus\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chshc\" (UniqueName: \"kubernetes.io/projected/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-kube-api-access-chshc\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.510974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwnjc\""
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-system-cni-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cni-binary-copy\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-hostroot\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-conf-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-daemon-config\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f5e3ce5-4e7a-416d-876d-88f5928a3755-iptables-alerter-script\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.510988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-modprobe-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.511353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.511916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.511823    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 21:09:43.512258 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.511936    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 21:09:43.512382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.512362    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.512821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.512795    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 21:09:43.512821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.512820    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lh2gw\""
Sep 10 21:09:43.512965 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.512855    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 21:09:43.513042 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 21:09:43.513122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 21:09:43.513216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513129    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.513216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.513593 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513489    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.513654 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.513626    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 21:09:43.514826 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.514650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-59p66\""
Sep 10 21:09:43.514826 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.514674    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.515998 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.515912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.516177 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.516154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:43.516326 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.516304    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 21:09:43.516506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.516488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ct6xd\""
Sep 10 21:09:43.516659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.516643    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 21:09:43.516947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.516853    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 21:09:43.522710 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.522689    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:09:43.529655 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.529615    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:09:43.529805 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.529672    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"
Sep 10 21:09:43.529805 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.529690    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal"]
Sep 10 21:09:43.535819 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.535771    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 21:04:43 +0000 UTC" deadline="2028-04-18 13:37:30.96301504 +0000 UTC"
Sep 10 21:09:43.535906 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.535820    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14056h27m47.427199329s"
Sep 10 21:09:43.542209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.542191    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal"]
Sep 10 21:09:43.542461 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.542442    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:09:43.606666 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.606644    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 21:09:43.611700 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-kubelet\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysconfig\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-run\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-var-lib-kubelet\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cnibin\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-hostroot\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f5e3ce5-4e7a-416d-876d-88f5928a3755-host-slash\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-run\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b1464-e117-4798-ae62-c4e8dbe02f75-agent-certs\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cnibin\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-var-lib-kubelet\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.611852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-conf-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysconfig\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b1464-e117-4798-ae62-c4e8dbe02f75-konnectivity-ca\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-hostroot\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f5e3ce5-4e7a-416d-876d-88f5928a3755-host-slash\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-conf-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-var-lib-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.611975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-netd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-sys-fs\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-host\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-host\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-bin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-bin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-env-overrides\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.612435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-socket-dir-parent\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-etc-kubernetes\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qvgz\" (UniqueName: \"kubernetes.io/projected/5f5e3ce5-4e7a-416d-876d-88f5928a3755-kube-api-access-8qvgz\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-log-socket\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-etc-kubernetes\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-bin\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-socket-dir-parent\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-config\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-tmp-dir\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cnibin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjltq\" (UniqueName: \"kubernetes.io/projected/ef6a4e86-206d-4dba-86cd-b033c338d8b3-kube-api-access-cjltq\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-etc-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cnibin\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kcg6\" (UniqueName: \"kubernetes.io/projected/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-kube-api-access-8kcg6\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-socket-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwd67\" (UniqueName: \"kubernetes.io/projected/a8efe91e-dab3-4f79-882b-3e8e13c07992-kube-api-access-nwd67\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-kubelet-config\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.612999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-netns\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-ovn\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxrrj\" (UniqueName: \"kubernetes.io/projected/4c1119f0-b7c3-478d-b539-96a81b1044bc-kube-api-access-hxrrj\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-slash\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-systemd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-kubelet-config\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-systemd\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-tmp-dir\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-systemd\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-hosts-file\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ng9v\" (UniqueName: \"kubernetes.io/projected/76912b7e-9cdd-4879-a340-e66ca7c0c79c-kube-api-access-2ng9v\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-hosts-file\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-multus\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.613790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-cni-multus\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-daemon-config\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chshc\" (UniqueName: \"kubernetes.io/projected/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-kube-api-access-chshc\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-system-cni-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cni-binary-copy\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f5e3ce5-4e7a-416d-876d-88f5928a3755-iptables-alerter-script\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76912b7e-9cdd-4879-a340-e66ca7c0c79c-system-cni-dir\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-node-log\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-device-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-modprobe-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-script-lib\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-modprobe-d\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.614695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-registration-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-conf\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-sys\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-cni-binary-copy\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f5e3ce5-4e7a-416d-876d-88f5928a3755-iptables-alerter-script\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.613951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-daemon-config\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r9db\" (UniqueName: \"kubernetes.io/projected/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-kube-api-access-7r9db\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8efe91e-dab3-4f79-882b-3e8e13c07992-host\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-sys\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-tuned\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.614319    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-sysctl-conf\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-system-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-k8s-cni-cncf-io\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-kubelet\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-k8s-cni-cncf-io\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.615401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-system-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-var-lib-kubelet\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.614553    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:44.1144807 +0000 UTC m=+1.993189767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8efe91e-dab3-4f79-882b-3e8e13c07992-serviceca\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-kubernetes\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-lib-modules\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-kubernetes\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-multus-cni-dir\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-dbus\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-dbus\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-lib-modules\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.614997    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dwjz\" (UniqueName: \"kubernetes.io/projected/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-kube-api-access-9dwjz\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-netns\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.616307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-multus-certs\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-systemd-units\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-multus-certs\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-host-run-netns\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-tmp\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.617209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.615922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76912b7e-9cdd-4879-a340-e66ca7c0c79c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.619118 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.619092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-etc-tuned\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.619208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.619156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-tmp\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.624165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.624144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qvgz\" (UniqueName: \"kubernetes.io/projected/5f5e3ce5-4e7a-416d-876d-88f5928a3755-kube-api-access-8qvgz\") pod \"iptables-alerter-sd2hh\" (UID: \"5f5e3ce5-4e7a-416d-876d-88f5928a3755\") " pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.624252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.624191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chshc\" (UniqueName: \"kubernetes.io/projected/2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc-kube-api-access-chshc\") pod \"tuned-8s8kk\" (UID: \"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc\") " pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.624890 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.624872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ng9v\" (UniqueName: \"kubernetes.io/projected/76912b7e-9cdd-4879-a340-e66ca7c0c79c-kube-api-access-2ng9v\") pod \"multus-additional-cni-plugins-6zwbc\" (UID: \"76912b7e-9cdd-4879-a340-e66ca7c0c79c\") " pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.628648 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.628630    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5n5pc"
Sep 10 21:09:43.631169 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.631139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dwjz\" (UniqueName: \"kubernetes.io/projected/a4040f5e-8b82-4b38-ab74-7e50328e2ab0-kube-api-access-9dwjz\") pod \"node-resolver-5zqd4\" (UID: \"a4040f5e-8b82-4b38-ab74-7e50328e2ab0\") " pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.632233 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.632217    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:43.632289 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.632236    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:43.632289 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.632246    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:43.632380 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.632320    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:44.132307062 +0000 UTC m=+2.011016104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:43.634972 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.634951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r9db\" (UniqueName: \"kubernetes.io/projected/fcfa4e0b-f081-4378-898f-8c5e737ed0c7-kube-api-access-7r9db\") pod \"multus-cbngr\" (UID: \"fcfa4e0b-f081-4378-898f-8c5e737ed0c7\") " pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.644541 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.644522    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5n5pc"
Sep 10 21:09:43.645477 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.645456    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf54620341eeda8eba6e62b7a0c0d35c.slice/crio-09c0e0dda86de7002e3192ce9b8a4592e069b278d9d9b8aa67591790bfd4390f WatchSource:0}: Error finding container 09c0e0dda86de7002e3192ce9b8a4592e069b278d9d9b8aa67591790bfd4390f: Status 404 returned error can't find the container with id 09c0e0dda86de7002e3192ce9b8a4592e069b278d9d9b8aa67591790bfd4390f
Sep 10 21:09:43.651582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.651567    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:09:43.697234 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.697212    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c9ae203a047aa285585b14707a01ad.slice/crio-d2484421ce7ce1324bdbd56a2130c74d2d8a7334721656cd08aa8c0c99a9ff88 WatchSource:0}: Error finding container d2484421ce7ce1324bdbd56a2130c74d2d8a7334721656cd08aa8c0c99a9ff88: Status 404 returned error can't find the container with id d2484421ce7ce1324bdbd56a2130c74d2d8a7334721656cd08aa8c0c99a9ff88
Sep 10 21:09:43.715825 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8efe91e-dab3-4f79-882b-3e8e13c07992-host\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.715912 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8efe91e-dab3-4f79-882b-3e8e13c07992-host\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.715912 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.715912 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8efe91e-dab3-4f79-882b-3e8e13c07992-serviceca\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.715912 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.716081 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-systemd-units\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716081 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716081 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.715989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-kubelet\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716081 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b1464-e117-4798-ae62-c4e8dbe02f75-agent-certs\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.716081 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-systemd-units\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-kubelet\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b1464-e117-4798-ae62-c4e8dbe02f75-konnectivity-ca\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-var-lib-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-netd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-sys-fs\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-env-overrides\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-netd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-var-lib-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-log-socket\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-sys-fs\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-bin\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-config\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-log-socket\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8efe91e-dab3-4f79-882b-3e8e13c07992-serviceca\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjltq\" (UniqueName: \"kubernetes.io/projected/ef6a4e86-206d-4dba-86cd-b033c338d8b3-kube-api-access-cjltq\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-cni-bin\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-etc-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kcg6\" (UniqueName: \"kubernetes.io/projected/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-kube-api-access-8kcg6\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-etc-openvswitch\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-socket-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwd67\" (UniqueName: \"kubernetes.io/projected/a8efe91e-dab3-4f79-882b-3e8e13c07992-kube-api-access-nwd67\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-netns\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-ovn\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxrrj\" (UniqueName: \"kubernetes.io/projected/4c1119f0-b7c3-478d-b539-96a81b1044bc-kube-api-access-hxrrj\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.716888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-slash\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-systemd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-node-log\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-env-overrides\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-device-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-script-lib\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b1464-e117-4798-ae62-c4e8dbe02f75-konnectivity-ca\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.716893    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-device-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-registration-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-registration-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:43.716976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:44.216952898 +0000 UTC m=+2.095662185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.716993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-netns\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-config\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.717641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-ovn\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-slash\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-run-systemd\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-socket-dir\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-node-log\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1119f0-b7c3-478d-b539-96a81b1044bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.718218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.717597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovnkube-script-lib\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.718279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.718506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.718481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b1464-e117-4798-ae62-c4e8dbe02f75-agent-certs\") pod \"konnectivity-agent-xl8x9\" (UID: \"e94b1464-e117-4798-ae62-c4e8dbe02f75\") " pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.726119 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.726095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwd67\" (UniqueName: \"kubernetes.io/projected/a8efe91e-dab3-4f79-882b-3e8e13c07992-kube-api-access-nwd67\") pod \"node-ca-fg6nr\" (UID: \"a8efe91e-dab3-4f79-882b-3e8e13c07992\") " pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.726419 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.726398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjltq\" (UniqueName: \"kubernetes.io/projected/ef6a4e86-206d-4dba-86cd-b033c338d8b3-kube-api-access-cjltq\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:43.726810 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.726794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kcg6\" (UniqueName: \"kubernetes.io/projected/1de57c84-3f66-4c7e-8815-e8beb0ac1fb4-kube-api-access-8kcg6\") pod \"ovnkube-node-dshvv\" (UID: \"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.726869 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.726846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxrrj\" (UniqueName: \"kubernetes.io/projected/4c1119f0-b7c3-478d-b539-96a81b1044bc-kube-api-access-hxrrj\") pod \"aws-ebs-csi-driver-node-bfs6f\" (UID: \"4c1119f0-b7c3-478d-b539-96a81b1044bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.813289 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.813225    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:43.834064 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.834043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8s8kk"
Sep 10 21:09:43.842055 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.842036    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e85c1c8_b8c8_4d4c_a6fd_7b06d43235dc.slice/crio-fdbbbec82d0295adb9f657cee790131301efb054728b07e51fc3c21364333859 WatchSource:0}: Error finding container fdbbbec82d0295adb9f657cee790131301efb054728b07e51fc3c21364333859: Status 404 returned error can't find the container with id fdbbbec82d0295adb9f657cee790131301efb054728b07e51fc3c21364333859
Sep 10 21:09:43.843816 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.843802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5zqd4"
Sep 10 21:09:43.850772 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.850752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4040f5e_8b82_4b38_ab74_7e50328e2ab0.slice/crio-b9cd9fdc1bff72db07140dff7a792a3cf89d4e8ab8b5769d619fed7189503f7c WatchSource:0}: Error finding container b9cd9fdc1bff72db07140dff7a792a3cf89d4e8ab8b5769d619fed7189503f7c: Status 404 returned error can't find the container with id b9cd9fdc1bff72db07140dff7a792a3cf89d4e8ab8b5769d619fed7189503f7c
Sep 10 21:09:43.856140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.856125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zwbc"
Sep 10 21:09:43.860086 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.860054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cbngr"
Sep 10 21:09:43.865143 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.865119    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76912b7e_9cdd_4879_a340_e66ca7c0c79c.slice/crio-47ded838d5b2c7d224576c4cb4c9b105a18384a89df42e91ff91a382e16b4df7 WatchSource:0}: Error finding container 47ded838d5b2c7d224576c4cb4c9b105a18384a89df42e91ff91a382e16b4df7: Status 404 returned error can't find the container with id 47ded838d5b2c7d224576c4cb4c9b105a18384a89df42e91ff91a382e16b4df7
Sep 10 21:09:43.868813 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.868776    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfa4e0b_f081_4378_898f_8c5e737ed0c7.slice/crio-d720c17cb0b3204d0b3aa5ed5a9a218446135a5336a939519a3689f5f2113cab WatchSource:0}: Error finding container d720c17cb0b3204d0b3aa5ed5a9a218446135a5336a939519a3689f5f2113cab: Status 404 returned error can't find the container with id d720c17cb0b3204d0b3aa5ed5a9a218446135a5336a939519a3689f5f2113cab
Sep 10 21:09:43.872053 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.872021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sd2hh"
Sep 10 21:09:43.879689 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.879670    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5e3ce5_4e7a_416d_876d_88f5928a3755.slice/crio-c2b195746d6cca591c1c9bd30182ad127e24988c200f87c645a95a8a51d0de46 WatchSource:0}: Error finding container c2b195746d6cca591c1c9bd30182ad127e24988c200f87c645a95a8a51d0de46: Status 404 returned error can't find the container with id c2b195746d6cca591c1c9bd30182ad127e24988c200f87c645a95a8a51d0de46
Sep 10 21:09:43.890565 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.890547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:09:43.895559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.895542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:09:43.898126 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.898109    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de57c84_3f66_4c7e_8815_e8beb0ac1fb4.slice/crio-f451367e81ec6404cb6effce7568ec0ce975498c3c2201159e302ff9d18c68f4 WatchSource:0}: Error finding container f451367e81ec6404cb6effce7568ec0ce975498c3c2201159e302ff9d18c68f4: Status 404 returned error can't find the container with id f451367e81ec6404cb6effce7568ec0ce975498c3c2201159e302ff9d18c68f4
Sep 10 21:09:43.903707 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.903686    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94b1464_e117_4798_ae62_c4e8dbe02f75.slice/crio-90b3c27e5d97af52fc96a258467f73c38a27ed8cdf564aa11017374749c88957 WatchSource:0}: Error finding container 90b3c27e5d97af52fc96a258467f73c38a27ed8cdf564aa11017374749c88957: Status 404 returned error can't find the container with id 90b3c27e5d97af52fc96a258467f73c38a27ed8cdf564aa11017374749c88957
Sep 10 21:09:43.919934 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.919906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f"
Sep 10 21:09:43.927320 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.927299    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1119f0_b7c3_478d_b539_96a81b1044bc.slice/crio-652662dbbee8275af00acdfb735c20f46238b5f42c6c7474f8863b4086daf274 WatchSource:0}: Error finding container 652662dbbee8275af00acdfb735c20f46238b5f42c6c7474f8863b4086daf274: Status 404 returned error can't find the container with id 652662dbbee8275af00acdfb735c20f46238b5f42c6c7474f8863b4086daf274
Sep 10 21:09:43.939987 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:43.939963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fg6nr"
Sep 10 21:09:43.946747 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:09:43.946725    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8efe91e_dab3_4f79_882b_3e8e13c07992.slice/crio-d7f79e9ef7f9bdae78062bb2893df75887b5fee6ffb1d875f523ce86b18d600e WatchSource:0}: Error finding container d7f79e9ef7f9bdae78062bb2893df75887b5fee6ffb1d875f523ce86b18d600e: Status 404 returned error can't find the container with id d7f79e9ef7f9bdae78062bb2893df75887b5fee6ffb1d875f523ce86b18d600e
Sep 10 21:09:44.119331 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.119251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:44.119442 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.119409    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:44.119489 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.119469    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:45.119454238 +0000 UTC m=+2.998163280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:44.220001 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.219964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:44.220165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.220019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:44.220215 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220206    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:44.220279 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220267    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:45.220247001 +0000 UTC m=+3.098956055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:44.220758 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220637    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:44.220758 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220669    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:44.220758 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220682    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:44.220758 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.220734    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:45.220717983 +0000 UTC m=+3.099427042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:44.619525 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.618864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:44.619525 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:44.618993    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:44.623206 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.623116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal" event={"ID":"cf54620341eeda8eba6e62b7a0c0d35c","Type":"ContainerStarted","Data":"09c0e0dda86de7002e3192ce9b8a4592e069b278d9d9b8aa67591790bfd4390f"}
Sep 10 21:09:44.629378 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.629304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f" event={"ID":"4c1119f0-b7c3-478d-b539-96a81b1044bc","Type":"ContainerStarted","Data":"652662dbbee8275af00acdfb735c20f46238b5f42c6c7474f8863b4086daf274"}
Sep 10 21:09:44.642749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.642719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"f451367e81ec6404cb6effce7568ec0ce975498c3c2201159e302ff9d18c68f4"}
Sep 10 21:09:44.646735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.646708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cbngr" event={"ID":"fcfa4e0b-f081-4378-898f-8c5e737ed0c7","Type":"ContainerStarted","Data":"d720c17cb0b3204d0b3aa5ed5a9a218446135a5336a939519a3689f5f2113cab"}
Sep 10 21:09:44.647518 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.647430    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:04:44 +0000 UTC" deadline="2028-02-15 06:17:35.277426097 +0000 UTC"
Sep 10 21:09:44.647518 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.647468    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12537h7m50.629963224s"
Sep 10 21:09:44.655583 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.655558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerStarted","Data":"47ded838d5b2c7d224576c4cb4c9b105a18384a89df42e91ff91a382e16b4df7"}
Sep 10 21:09:44.676459 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.676253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8s8kk" event={"ID":"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc","Type":"ContainerStarted","Data":"fdbbbec82d0295adb9f657cee790131301efb054728b07e51fc3c21364333859"}
Sep 10 21:09:44.690151 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.690122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fg6nr" event={"ID":"a8efe91e-dab3-4f79-882b-3e8e13c07992","Type":"ContainerStarted","Data":"d7f79e9ef7f9bdae78062bb2893df75887b5fee6ffb1d875f523ce86b18d600e"}
Sep 10 21:09:44.704622 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.704594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xl8x9" event={"ID":"e94b1464-e117-4798-ae62-c4e8dbe02f75","Type":"ContainerStarted","Data":"90b3c27e5d97af52fc96a258467f73c38a27ed8cdf564aa11017374749c88957"}
Sep 10 21:09:44.716438 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.716288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sd2hh" event={"ID":"5f5e3ce5-4e7a-416d-876d-88f5928a3755","Type":"ContainerStarted","Data":"c2b195746d6cca591c1c9bd30182ad127e24988c200f87c645a95a8a51d0de46"}
Sep 10 21:09:44.736275 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.736215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5zqd4" event={"ID":"a4040f5e-8b82-4b38-ab74-7e50328e2ab0","Type":"ContainerStarted","Data":"b9cd9fdc1bff72db07140dff7a792a3cf89d4e8ab8b5769d619fed7189503f7c"}
Sep 10 21:09:44.739907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.739854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal" event={"ID":"52c9ae203a047aa285585b14707a01ad","Type":"ContainerStarted","Data":"d2484421ce7ce1324bdbd56a2130c74d2d8a7334721656cd08aa8c0c99a9ff88"}
Sep 10 21:09:44.810115 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:44.810071    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:45.126498 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.126423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:45.126696 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.126615    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:45.126696 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.126679    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:47.126659002 +0000 UTC m=+5.005368060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.226907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.227003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227144    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227166    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227179    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227233    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:47.227215661 +0000 UTC m=+5.105924717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227648    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:45.227731 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.227699    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:47.22768332 +0000 UTC m=+5.106392368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:45.605984 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.605150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:45.605984 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.605314    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:45.605984 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.605762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:45.605984 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:45.605881    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:45.610150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.610128    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:09:45.648106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.648061    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:04:44 +0000 UTC" deadline="2028-06-08 06:00:09.827375692 +0000 UTC"
Sep 10 21:09:45.648106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:45.648104    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15272h50m24.179275952s"
Sep 10 21:09:46.606946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:46.606915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:46.607123 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:46.607041    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:47.146984 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:47.146902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:47.147597 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.147167    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:47.147597 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.147241    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:51.147220692 +0000 UTC m=+9.025929741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:47.248231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:47.248285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.248464    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.248525    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:51.248505874 +0000 UTC m=+9.127214932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.248959    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.248981    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.248994    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:47.249113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.249041    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:51.249025844 +0000 UTC m=+9.127734891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:47.605201 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:47.605165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:47.605420 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.605389    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:47.605518 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:47.605441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:47.605582 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:47.605563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:48.605195 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:48.605038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:48.605195 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:48.605188    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:49.605473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:49.605440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:49.605952 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:49.605580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:49.606034 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:49.605968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:49.606094 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:49.606057    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:50.605816 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:50.605777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:50.606223 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:50.605930    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:51.183323 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:51.182745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:51.183323 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.182899    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:51.183323 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.182972    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:59.182951416 +0000 UTC m=+17.061660463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:51.283661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:51.283626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:51.283825 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:51.283677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:51.283825 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283782    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:51.283825 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283787    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:51.283825 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283809    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:51.283825 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283822    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:51.284113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283840    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:59.283821646 +0000 UTC m=+17.162530700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:51.284113 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.283858    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:09:59.283847898 +0000 UTC m=+17.162556962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:51.605614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:51.605525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:51.605804 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:51.605522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:51.605804 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.605661    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:51.605804 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:51.605774    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:52.606568 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:52.606458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:52.611712 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:52.607792    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:53.604784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:53.604753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:53.604784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:53.604773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:53.605191 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:53.604890    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:53.605191 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:53.604984    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:54.605117 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:54.605089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:54.605626 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:54.605215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:55.605165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:55.605137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:55.605532 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:55.605143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:55.605532 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:55.605236    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:55.605532 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:55.605332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:56.605578 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:56.605542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:56.606034 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:56.605687    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:57.605143 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:57.605110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:57.605319 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:57.605229    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:57.605319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:57.605297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:57.605491 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:57.605410    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:09:58.605400 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:58.605368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:58.605838 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:58.605508    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:09:59.240465 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:59.240432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:09:59.240639 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.240552    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:59.240639 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.240607    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:15.240592325 +0000 UTC m=+33.119301368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:09:59.341663 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:59.341616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:59.341834 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:59.341672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:59.341834 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341807    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:59.341834 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341813    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:09:59.342001 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341839    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:09:59.342001 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341855    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:59.342001 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341874    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:15.341854042 +0000 UTC m=+33.220563104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:09:59.342001 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.341914    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:15.34189578 +0000 UTC m=+33.220604856 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:09:59.605482 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:59.605404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:09:59.605876 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.605550    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:09:59.605876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:09:59.605597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:09:59.605876 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:09:59.605681    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:00.608534 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:00.608504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:00.608999 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:00.608618    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:01.605790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:01.605757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:01.605961 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:01.605878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:01.605961 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:01.605934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:01.606069 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:01.606044    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:02.605613 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:02.605580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:02.606101 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:02.605690    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:03.604997 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.604757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:03.605119 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:03.605094    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:03.605182 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.604754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:03.605236 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:03.605189    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:03.795526 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.795496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5zqd4" event={"ID":"a4040f5e-8b82-4b38-ab74-7e50328e2ab0","Type":"ContainerStarted","Data":"510e2b0a7ec20c3859b6fffb6000f3fa8bee3f366bb5e96ac0395acbd8250bc0"}
Sep 10 21:10:03.797004 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.796979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal" event={"ID":"52c9ae203a047aa285585b14707a01ad","Type":"ContainerStarted","Data":"5823985a7d1509572a48bce812c68c5cde3a1ee8c27cf9f448ff90233f379461"}
Sep 10 21:10:03.798399 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.798379    2580 generic.go:356] "Generic (PLEG): container finished" podID="cf54620341eeda8eba6e62b7a0c0d35c" containerID="5e6464586f50590ebd0b0e945361624cfed5f91e1a8c644debc62ff07782cf79" exitCode=0
Sep 10 21:10:03.798530 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.798424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal" event={"ID":"cf54620341eeda8eba6e62b7a0c0d35c","Type":"ContainerDied","Data":"5e6464586f50590ebd0b0e945361624cfed5f91e1a8c644debc62ff07782cf79"}
Sep 10 21:10:03.800021 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.799993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f" event={"ID":"4c1119f0-b7c3-478d-b539-96a81b1044bc","Type":"ContainerStarted","Data":"e4c1c211007da014a093b6181e8ecc2921d09b9eda4d5934571b930ab9df3046"}
Sep 10 21:10:03.802691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.802674    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:10:03.802956 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.802939    2580 generic.go:356] "Generic (PLEG): container finished" podID="1de57c84-3f66-4c7e-8815-e8beb0ac1fb4" containerID="62379f0875335b274f7751e570003bf1390fbc5a438327e6a9eb96ffcbc098f1" exitCode=1
Sep 10 21:10:03.803028 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"0deebff4cc05e584702863a28b8e67f232ec4518136be539b934a6d650135bec"}
Sep 10 21:10:03.803079 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"fe734f9d91ae9982b01fdb690cd2205425665f6f586e0ac870373a0daaa914cf"}
Sep 10 21:10:03.803123 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"cdeb0fc99f26eb91b92737573d86096f7ea94996e4d6cd30682f0358008de5c0"}
Sep 10 21:10:03.803179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"79fcf86e4c1819c1c49ac33c86ff1e97d481813e2ff8dd251bdbdb478e904d51"}
Sep 10 21:10:03.803179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerDied","Data":"62379f0875335b274f7751e570003bf1390fbc5a438327e6a9eb96ffcbc098f1"}
Sep 10 21:10:03.803266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.803209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"576ab92e424cff18377c0c8f1258d8b17c64777d4a0a5f960e3fed5532fbe9e6"}
Sep 10 21:10:03.804319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.804297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cbngr" event={"ID":"fcfa4e0b-f081-4378-898f-8c5e737ed0c7","Type":"ContainerStarted","Data":"a4cac9cf7dea02f7271a0bf5fed251e7fc667da85b269d40a98ae6c48b5559b9"}
Sep 10 21:10:03.805600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.805582    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="5f6e630ee8e9db4dad966271b9c9568836a2d1ee950c805de8e5359cadd2a33b" exitCode=0
Sep 10 21:10:03.805675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.805633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"5f6e630ee8e9db4dad966271b9c9568836a2d1ee950c805de8e5359cadd2a33b"}
Sep 10 21:10:03.807068 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.807053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8s8kk" event={"ID":"2e85c1c8-b8c8-4d4c-a6fd-7b06d43235dc","Type":"ContainerStarted","Data":"48a9750f249758760e4a582f302660997069af84babbfc1651b018e1610595f9"}
Sep 10 21:10:03.808477 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.808460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fg6nr" event={"ID":"a8efe91e-dab3-4f79-882b-3e8e13c07992","Type":"ContainerStarted","Data":"54c4a91d5136e02c4ae6327941332be438578594b2ad6bf08696184afc6f2bfc"}
Sep 10 21:10:03.809758 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.809738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xl8x9" event={"ID":"e94b1464-e117-4798-ae62-c4e8dbe02f75","Type":"ContainerStarted","Data":"a8bf5940c5e214c714bf247fa685cb6e58f9a20937022823b5080fb9bd28facc"}
Sep 10 21:10:03.821732 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.821696    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5zqd4" podStartSLOduration=1.794124461 podStartE2EDuration="20.821684777s" totalImagesPullingTime="19.027560316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.821412079 +0000 UTC m=+21.700121143" watchObservedRunningTime="2026-09-10 21:10:03.821684777 +0000 UTC m=+21.700393838"
Sep 10 21:10:03.835214 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.835175    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xl8x9" podStartSLOduration=1.862976711 podStartE2EDuration="20.835163469s" totalImagesPullingTime="18.972186758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.834894987 +0000 UTC m=+21.713604050" watchObservedRunningTime="2026-09-10 21:10:03.835163469 +0000 UTC m=+21.713872533"
Sep 10 21:10:03.848655 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.848614    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fg6nr" podStartSLOduration=5.833055599 podStartE2EDuration="20.848597908s" totalImagesPullingTime="15.015542309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.848183864 +0000 UTC m=+21.726892926" watchObservedRunningTime="2026-09-10 21:10:03.848597908 +0000 UTC m=+21.727306984"
Sep 10 21:10:03.876230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.875729    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-178.ec2.internal" podStartSLOduration=19.875713807 podStartE2EDuration="19.875713807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.874800796 +0000 UTC m=+21.753509860" watchObservedRunningTime="2026-09-10 21:10:03.875713807 +0000 UTC m=+21.754422870"
Sep 10 21:10:03.894226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.894194    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8s8kk" podStartSLOduration=1.856410687 podStartE2EDuration="20.89418331s" totalImagesPullingTime="19.037772623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.894080721 +0000 UTC m=+21.772789783" watchObservedRunningTime="2026-09-10 21:10:03.89418331 +0000 UTC m=+21.772892372"
Sep 10 21:10:03.937668 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:03.937629    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cbngr" podStartSLOduration=1.6783506689999999 podStartE2EDuration="20.937615447s" totalImagesPullingTime="19.259264778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:03.936193373 +0000 UTC m=+21.814902435" watchObservedRunningTime="2026-09-10 21:10:03.937615447 +0000 UTC m=+21.816324509"
Sep 10 21:10:04.605400 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:04.605368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:04.605558 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:04.605501    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:04.814138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:04.813913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sd2hh" event={"ID":"5f5e3ce5-4e7a-416d-876d-88f5928a3755","Type":"ContainerStarted","Data":"f8131684ef69b3f50ef10c1a0c8eb728c9b2f98b5b7e178ecb788c6e112c3069"}
Sep 10 21:10:04.816384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:04.816319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal" event={"ID":"cf54620341eeda8eba6e62b7a0c0d35c","Type":"ContainerStarted","Data":"271d4bc6d69ba8dc586b567dcead57004360fd081a9b6f25d07b26c8f087f84a"}
Sep 10 21:10:04.829951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:04.829655    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sd2hh" podStartSLOduration=2.845366088 podStartE2EDuration="21.82963794s" totalImagesPullingTime="18.984271852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:04.828950034 +0000 UTC m=+22.707659097" watchObservedRunningTime="2026-09-10 21:10:04.82963794 +0000 UTC m=+22.708347002"
Sep 10 21:10:04.845739 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:04.845691    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-178.ec2.internal" podStartSLOduration=20.845670693 podStartE2EDuration="20.845670693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:04.844781446 +0000 UTC m=+22.723490510" watchObservedRunningTime="2026-09-10 21:10:04.845670693 +0000 UTC m=+22.724379756"
Sep 10 21:10:05.094961 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.094817    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 21:10:05.563911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.563745    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T21:10:05.094958002Z","UUID":"1082ef90-b9cc-42a3-824a-1cca6791c00b","Handler":null,"Name":"","Endpoint":""}
Sep 10 21:10:05.565137 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.565122    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 21:10:05.565205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.565157    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 21:10:05.605130 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.605099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:05.605265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.605128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:05.605265 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:05.605231    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:05.605404 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:05.605336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:05.820157 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:05.820131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f" event={"ID":"4c1119f0-b7c3-478d-b539-96a81b1044bc","Type":"ContainerStarted","Data":"0f61449ccf24ddea450ba08d7b77b183d217785cce480ff263043956b7abb0d6"}
Sep 10 21:10:06.605460 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:06.605436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:06.605579 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:06.605558    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:06.824314 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:06.824217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f" event={"ID":"4c1119f0-b7c3-478d-b539-96a81b1044bc","Type":"ContainerStarted","Data":"0967445fed4399a597da98a1e8a09bb4cb5a5d683bac6a70fa2781c72e0a5fb8"}
Sep 10 21:10:06.827320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:06.827302    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:10:06.827710 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:06.827680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"221fe1344ad3e8f1c864cc87f6022a8b41b693c67a26da7957e2eec3ff0df060"}
Sep 10 21:10:06.845773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:06.845735    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bfs6f" podStartSLOduration=2.145693019 podStartE2EDuration="23.845722875s" totalImagesPullingTime="21.700029856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:06.845570923 +0000 UTC m=+24.724279986" watchObservedRunningTime="2026-09-10 21:10:06.845722875 +0000 UTC m=+24.724431937"
Sep 10 21:10:07.605795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:07.605231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:07.605795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:07.605445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:07.605795 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:07.605402    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:07.606292 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:07.606115    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:08.177241 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.177204    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:10:08.177822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.177798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:10:08.605071 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.604992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:08.605205 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:08.605105    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:08.836812 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.836720    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:10:08.837296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"a59b640016626985cdd87020b945fdd6307a6fb5a2b8752db87acd0b45709661"}
Sep 10 21:10:08.837489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:10:08.837489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837407    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:08.837489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:08.837630 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:08.837835 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.837631    2580 scope.go:117] "RemoveContainer" containerID="62379f0875335b274f7751e570003bf1390fbc5a438327e6a9eb96ffcbc098f1"
Sep 10 21:10:08.838444 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.838064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xl8x9"
Sep 10 21:10:08.856774 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.856639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:08.857739 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:08.857681    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:09.605646 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.605476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:09.605987 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.605531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:09.605987 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:09.605714    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:09.605987 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:09.605788    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:09.842500 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.842475    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:10:09.842881 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.842857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" event={"ID":"1de57c84-3f66-4c7e-8815-e8beb0ac1fb4","Type":"ContainerStarted","Data":"a1ecd4a72d541ff594609d3153113268dd10cc2fc5537353ec9ae730684c7c3a"}
Sep 10 21:10:09.844574 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.844552    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="db6ea051e003b8c57d17998357a9cb7b6e012d1df6a7b3a23a1cfaa30a3eb3dd" exitCode=0
Sep 10 21:10:09.844659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.844629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"db6ea051e003b8c57d17998357a9cb7b6e012d1df6a7b3a23a1cfaa30a3eb3dd"}
Sep 10 21:10:09.899919 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:09.899884    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv" podStartSLOduration=7.64517913 podStartE2EDuration="26.899870956s" totalImagesPullingTime="19.254691826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:09.877336229 +0000 UTC m=+27.756045290" watchObservedRunningTime="2026-09-10 21:10:09.899870956 +0000 UTC m=+27.778580009"
Sep 10 21:10:10.605328 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.605298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:10.605513 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:10.605440    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:10.849069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.849038    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="4999f47028668fdaef5381cf861ca68de2afceff07f369ca05765b6be5540b37" exitCode=0
Sep 10 21:10:10.849721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.849119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"4999f47028668fdaef5381cf861ca68de2afceff07f369ca05765b6be5540b37"}
Sep 10 21:10:10.886428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.886402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p2vbb"]
Sep 10 21:10:10.886530 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.886512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:10.886638 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:10.886621    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:10.886998 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.886978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kjzvc"]
Sep 10 21:10:10.887067 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.887055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:10.887169 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:10.887152    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:10.893052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.893033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4rf6f"]
Sep 10 21:10:10.893136 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:10.893109    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:10.893192 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:10.893176    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:11.853747 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:11.853655    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="88e0d0fdbf05d88700b62486c3cc5cafb6b5902ee61bdfa9498eec04843f1319" exitCode=0
Sep 10 21:10:11.853747 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:11.853726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"88e0d0fdbf05d88700b62486c3cc5cafb6b5902ee61bdfa9498eec04843f1319"}
Sep 10 21:10:12.606047 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:12.605814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:12.606047 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:12.605911    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:12.606047 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:12.605962    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:12.606297 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:12.606075    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:12.606297 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:12.606110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:12.606297 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:12.606197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:14.605344 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:14.605311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:14.605775 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:14.605311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:14.605775 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:14.605411    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4rf6f" podUID="4f27baec-6902-4b28-ae58-0d4f103402d6"
Sep 10 21:10:14.605775 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:14.605322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:14.605775 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:14.605485    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjzvc" podUID="0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668"
Sep 10 21:10:14.605775 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:14.605555    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2vbb" podUID="ef6a4e86-206d-4dba-86cd-b033c338d8b3"
Sep 10 21:10:15.266068 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.266038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:15.266221 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.266170    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:15.266258 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.266226    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret podName:0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.266211939 +0000 UTC m=+65.144920996 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret") pod "global-pull-secret-syncer-kjzvc" (UID: "0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:15.366366 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.366334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:15.366505 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.366373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:15.366505 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366493    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:15.366622 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366513    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:15.366622 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366535    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:15.366622 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366546    2580 projected.go:196] Error preparing data for projected volume kube-api-access-gdg4b for pod openshift-network-diagnostics/network-check-target-4rf6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:15.366622 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366551    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.366534454 +0000 UTC m=+65.245243514 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:15.366622 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:15.366589    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b podName:4f27baec-6902-4b28-ae58-0d4f103402d6 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.366575733 +0000 UTC m=+65.245284792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gdg4b" (UniqueName: "kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b") pod "network-check-target-4rf6f" (UID: "4f27baec-6902-4b28-ae58-0d4f103402d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:15.902292 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.902241    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-178.ec2.internal" event="NodeReady"
Sep 10 21:10:15.902718 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.902421    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 21:10:15.941568 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.941489    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65548dd84b-nw87k"]
Sep 10 21:10:15.973928 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.973899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bpg2v"]
Sep 10 21:10:15.974034 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.974006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:15.976781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.976490    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 21:10:15.976781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.976591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pjnbl\""
Sep 10 21:10:15.976781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.976625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 21:10:15.976781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.976625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 21:10:15.984426 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.984405    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 21:10:15.990097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.990075    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65548dd84b-nw87k"]
Sep 10 21:10:15.990188 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.990106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:15.990188 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.990141    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b8h6t"]
Sep 10 21:10:15.992548 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.992515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 21:10:15.992661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.992555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 21:10:15.992661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.992643    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2h9gs\""
Sep 10 21:10:15.992763 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:15.992690    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 21:10:16.012210 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.012126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpg2v"]
Sep 10 21:10:16.012210 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.012172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b8h6t"]
Sep 10 21:10:16.012365 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.012319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.015617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.015573    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 21:10:16.015724 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.015707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:10:16.016993 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.016974    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 21:10:16.071477 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071576 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc6r4\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071576 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071576 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071576 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/714039b9-eb08-4aa6-90c9-260cfb76b60c-cert\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.071735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnfg\" (UniqueName: \"kubernetes.io/projected/714039b9-eb08-4aa6-90c9-260cfb76b60c-kube-api-access-wtnfg\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.071735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.071894 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.071750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.144436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.144392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:16.172321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172434 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172434 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/550f50e1-7b22-4f80-809a-43044ae513ac-tmp-dir\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.172434 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/550f50e1-7b22-4f80-809a-43044ae513ac-config-volume\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.172555 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172604 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172658 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc6r4\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172706 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172706 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.172688    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:16.172798 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.172706    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:16.172798 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.172762    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:16.672742288 +0000 UTC m=+34.551451341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:16.172934 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.172986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.173079 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.172989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/714039b9-eb08-4aa6-90c9-260cfb76b60c-cert\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.173079 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.173062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnfg\" (UniqueName: \"kubernetes.io/projected/714039b9-eb08-4aa6-90c9-260cfb76b60c-kube-api-access-wtnfg\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.173179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.173083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.173179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.173091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.173179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.173106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zrtz\" (UniqueName: \"kubernetes.io/projected/550f50e1-7b22-4f80-809a-43044ae513ac-kube-api-access-8zrtz\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.173838 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.173815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.177407 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.177366    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.177407 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.177381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.177407 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.177390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/714039b9-eb08-4aa6-90c9-260cfb76b60c-cert\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.186553 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.186455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnfg\" (UniqueName: \"kubernetes.io/projected/714039b9-eb08-4aa6-90c9-260cfb76b60c-kube-api-access-wtnfg\") pod \"ingress-canary-bpg2v\" (UID: \"714039b9-eb08-4aa6-90c9-260cfb76b60c\") " pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.186649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.186599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.186710 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.186660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc6r4\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.241294 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.241201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:16.274179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.274305 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zrtz\" (UniqueName: \"kubernetes.io/projected/550f50e1-7b22-4f80-809a-43044ae513ac-kube-api-access-8zrtz\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.274305 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/550f50e1-7b22-4f80-809a-43044ae513ac-tmp-dir\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.274305 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/550f50e1-7b22-4f80-809a-43044ae513ac-config-volume\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.274509 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.274469    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:16.274631 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.274533    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:16.774516439 +0000 UTC m=+34.653225496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:16.274695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/550f50e1-7b22-4f80-809a-43044ae513ac-tmp-dir\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.274794 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.274775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/550f50e1-7b22-4f80-809a-43044ae513ac-config-volume\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.282420 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.282399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zrtz\" (UniqueName: \"kubernetes.io/projected/550f50e1-7b22-4f80-809a-43044ae513ac-kube-api-access-8zrtz\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.299374 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.299350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpg2v"
Sep 10 21:10:16.605528 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.605455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:16.605528 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.605496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:16.605791 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.605455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:16.607969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.607947    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:10:16.608807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.608659    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:10:16.608807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.608672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:10:16.608807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.608706    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:10:16.608807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.608666    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:10:16.608807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.608747    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:10:16.677115 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.677090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:16.677291 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.677236    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:16.677291 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.677251    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:16.677411 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.677324    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:17.677304976 +0000 UTC m=+35.556014040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:16.777621 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:16.777590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:16.777814 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.777757    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:16.777913 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:16.777886    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:17.777857614 +0000 UTC m=+35.656566682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:17.197365 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.197342    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpg2v"]
Sep 10 21:10:17.201505 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:10:17.201482    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714039b9_eb08_4aa6_90c9_260cfb76b60c.slice/crio-f97b2351ee1773e1b20906fadaa07423828dba138e7d68870aec58be7cc29365 WatchSource:0}: Error finding container f97b2351ee1773e1b20906fadaa07423828dba138e7d68870aec58be7cc29365: Status 404 returned error can't find the container with id f97b2351ee1773e1b20906fadaa07423828dba138e7d68870aec58be7cc29365
Sep 10 21:10:17.682616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.682450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:17.682616 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:17.682609    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:17.682832 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:17.682629    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:17.682832 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:17.682697    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:19.682674888 +0000 UTC m=+37.561383929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:17.783664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.783634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:17.783787 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:17.783772    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:17.783838 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:17.783828    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:19.783814908 +0000 UTC m=+37.662523947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:17.871440 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.871411    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="d5a6f89a9e8dbe06b40aa5b38095297bae00eb84475fc7f73c68722ae299d063" exitCode=0
Sep 10 21:10:17.871588 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.871496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"d5a6f89a9e8dbe06b40aa5b38095297bae00eb84475fc7f73c68722ae299d063"}
Sep 10 21:10:17.872790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:17.872759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpg2v" event={"ID":"714039b9-eb08-4aa6-90c9-260cfb76b60c","Type":"ContainerStarted","Data":"f97b2351ee1773e1b20906fadaa07423828dba138e7d68870aec58be7cc29365"}
Sep 10 21:10:18.877675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:18.877639    2580 generic.go:356] "Generic (PLEG): container finished" podID="76912b7e-9cdd-4879-a340-e66ca7c0c79c" containerID="f83aff5da41726bb2e6366d59857921b91b4388cfa56769549b35157d81c0963" exitCode=0
Sep 10 21:10:18.878208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:18.877696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerDied","Data":"f83aff5da41726bb2e6366d59857921b91b4388cfa56769549b35157d81c0963"}
Sep 10 21:10:19.700044 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:19.700010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:19.700201 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:19.700169    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:19.700201 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:19.700189    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:19.700279 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:19.700244    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:23.700228532 +0000 UTC m=+41.578937573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:19.800469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:19.800434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:19.800605 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:19.800556    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:19.800651 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:19.800621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:23.800609035 +0000 UTC m=+41.679318087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:19.883868 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:19.883840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" event={"ID":"76912b7e-9cdd-4879-a340-e66ca7c0c79c","Type":"ContainerStarted","Data":"a701e138ec383e93151658265bb73a5ebd12c3892a9ae97ff6cf808610065205"}
Sep 10 21:10:19.904809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:19.904770    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6zwbc" podStartSLOduration=6.042030728 podStartE2EDuration="36.904756893s" totalImagesPullingTime="30.862726165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:19.903373795 +0000 UTC m=+37.782082856" watchObservedRunningTime="2026-09-10 21:10:19.904756893 +0000 UTC m=+37.783465946"
Sep 10 21:10:20.024467 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:20.024436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:20.125910 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:20.125851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:20.887344 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:20.887305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpg2v" event={"ID":"714039b9-eb08-4aa6-90c9-260cfb76b60c","Type":"ContainerStarted","Data":"6d97dabe9e8e7e60b02c801b6e95d9304ee97878735d6d5afa1751fd706e53a3"}
Sep 10 21:10:20.903884 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:20.903842    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bpg2v" podStartSLOduration=3.175290631 podStartE2EDuration="5.903828928s" totalImagesPullingTime="2.728538297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:20.902438705 +0000 UTC m=+38.781147766" watchObservedRunningTime="2026-09-10 21:10:20.903828928 +0000 UTC m=+38.782538025"
Sep 10 21:10:21.166071 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:21.166005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:21.227704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:21.227676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:23.729744 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:23.729698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:23.730139 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:23.729847    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:23.730139 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:23.729867    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:23.730139 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:23.729938    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:31.729923954 +0000 UTC m=+49.608632997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:23.830462 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:23.830432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:23.830613 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:23.830576    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:23.830654 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:23.830632    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:31.830619554 +0000 UTC m=+49.709328594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:24.827131 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:24.827100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:24.924132 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:24.924104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:25.030356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:25.030323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:25.123309 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:25.123226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:25.226244 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:25.226217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:25.330023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:25.329996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:25.934823 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:25.934792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:26.035450 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:26.035424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:26.338180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:26.338111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:26.398605 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:26.398582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:31.781767 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:31.781735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:31.782144 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:31.781834    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:31.782144 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:31.781846    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:31.782144 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:31.781915    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.78190016 +0000 UTC m=+65.660609213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:31.882854 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:31.882825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:31.882956 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:31.882941    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:31.882999 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:31.882987    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.882975678 +0000 UTC m=+65.761684717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:40.859717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:40.859689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dshvv"
Sep 10 21:10:47.284084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.284049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:47.286562 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.286542    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:10:47.298140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.298111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668-original-pull-secret\") pod \"global-pull-secret-syncer-kjzvc\" (UID: \"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668\") " pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:47.384478 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.384451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:47.384588 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.384482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:10:47.386936 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.386915    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:10:47.387004 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.386983    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:10:47.395536 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.395515    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 21:10:47.395607 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.395586    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs podName:ef6a4e86-206d-4dba-86cd-b033c338d8b3 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:51.395572367 +0000 UTC m=+129.274281419 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs") pod "network-metrics-daemon-p2vbb" (UID: "ef6a4e86-206d-4dba-86cd-b033c338d8b3") : secret "metrics-daemon-secret" not found
Sep 10 21:10:47.397320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.397307    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:10:47.408219 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.408194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdg4b\" (UniqueName: \"kubernetes.io/projected/4f27baec-6902-4b28-ae58-0d4f103402d6-kube-api-access-gdg4b\") pod \"network-check-target-4rf6f\" (UID: \"4f27baec-6902-4b28-ae58-0d4f103402d6\") " pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:47.525238 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.525215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjzvc"
Sep 10 21:10:47.532712 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.532692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:10:47.540579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.540533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:47.787923 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.787886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:10:47.788064 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.788025    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:10:47.788064 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.788041    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:10:47.788153 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.788093    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:19.788079367 +0000 UTC m=+97.666788407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:10:47.818629 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.818580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kjzvc"]
Sep 10 21:10:47.821750 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:10:47.821725    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc55b83_b0f4_4f7f_a39e_2aae6cb8c668.slice/crio-8c1461b194bfec2599434265b83dcbc4c36002e30891cca9143ec5286bf6e8f5 WatchSource:0}: Error finding container 8c1461b194bfec2599434265b83dcbc4c36002e30891cca9143ec5286bf6e8f5: Status 404 returned error can't find the container with id 8c1461b194bfec2599434265b83dcbc4c36002e30891cca9143ec5286bf6e8f5
Sep 10 21:10:47.842471 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.842449    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4rf6f"]
Sep 10 21:10:47.845490 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:10:47.845469    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f27baec_6902_4b28_ae58_0d4f103402d6.slice/crio-99fa5f26e8ae5ad9a1ee073c71abb39ff49fb49e1fe94b9418f26e4d78390a55 WatchSource:0}: Error finding container 99fa5f26e8ae5ad9a1ee073c71abb39ff49fb49e1fe94b9418f26e4d78390a55: Status 404 returned error can't find the container with id 99fa5f26e8ae5ad9a1ee073c71abb39ff49fb49e1fe94b9418f26e4d78390a55
Sep 10 21:10:47.888943 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.888919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:10:47.889047 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.889035    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:47.889094 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:10:47.889083    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:11:19.889070008 +0000 UTC m=+97.767779049 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:47.952346 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.952313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4rf6f" event={"ID":"4f27baec-6902-4b28-ae58-0d4f103402d6","Type":"ContainerStarted","Data":"99fa5f26e8ae5ad9a1ee073c71abb39ff49fb49e1fe94b9418f26e4d78390a55"}
Sep 10 21:10:47.953362 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:47.953340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kjzvc" event={"ID":"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668","Type":"ContainerStarted","Data":"8c1461b194bfec2599434265b83dcbc4c36002e30891cca9143ec5286bf6e8f5"}
Sep 10 21:10:52.967896 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:52.967862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4rf6f" event={"ID":"4f27baec-6902-4b28-ae58-0d4f103402d6","Type":"ContainerStarted","Data":"82e60f9bd5d060403eae75f05c5884373a51608d9737c96b0b99213b0d0370c6"}
Sep 10 21:10:52.968387 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:52.967984    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:10:52.969345 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:52.969324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kjzvc" event={"ID":"0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668","Type":"ContainerStarted","Data":"555c5b0d4b5d729909837edd9e80058f9db5c6fede5685adf31c2fc59fc0bd85"}
Sep 10 21:10:52.999218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:52.999179    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4rf6f" podStartSLOduration=65.335153283 podStartE2EDuration="1m9.999167016s" totalImagesPullingTime="4.664013733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:52.985154848 +0000 UTC m=+70.863863911" watchObservedRunningTime="2026-09-10 21:10:52.999167016 +0000 UTC m=+70.877876080"
Sep 10 21:10:52.999362 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:10:52.999342    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kjzvc" podStartSLOduration=65.308864314 podStartE2EDuration="1m9.999335885s" totalImagesPullingTime="4.690471571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:52.998913295 +0000 UTC m=+70.877622356" watchObservedRunningTime="2026-09-10 21:10:52.999335885 +0000 UTC m=+70.878044944"
Sep 10 21:11:19.797083 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:19.797032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") pod \"image-registry-65548dd84b-nw87k\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") " pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:11:19.797577 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:19.797199    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 21:11:19.797577 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:19.797214    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65548dd84b-nw87k: secret "image-registry-tls" not found
Sep 10 21:11:19.797577 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:19.797321    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls podName:352ace2f-9c0c-4470-8aec-b9a98e1d9fa2 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:23.797263669 +0000 UTC m=+161.675972729 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls") pod "image-registry-65548dd84b-nw87k" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2") : secret "image-registry-tls" not found
Sep 10 21:11:19.897684 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:19.897655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:11:19.897779 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:19.897767    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:19.897837 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:19.897828    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls podName:550f50e1-7b22-4f80-809a-43044ae513ac nodeName:}" failed. No retries permitted until 2026-09-10 21:12:23.897812447 +0000 UTC m=+161.776521504 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls") pod "dns-default-b8h6t" (UID: "550f50e1-7b22-4f80-809a-43044ae513ac") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:22.410373 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.410343    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"]
Sep 10 21:11:22.414136 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.414120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"
Sep 10 21:11:22.416853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.416829    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.416980 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.416960    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qrxpc\""
Sep 10 21:11:22.417514 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.417499    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.425664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.425642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"]
Sep 10 21:11:22.515052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.515011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjlh\" (UniqueName: \"kubernetes.io/projected/2621728c-025f-489c-a811-97f005237bc1-kube-api-access-fjjlh\") pod \"volume-data-source-validator-86695b7cc7-8tsbz\" (UID: \"2621728c-025f-489c-a811-97f005237bc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"
Sep 10 21:11:22.615697 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.615669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjlh\" (UniqueName: \"kubernetes.io/projected/2621728c-025f-489c-a811-97f005237bc1-kube-api-access-fjjlh\") pod \"volume-data-source-validator-86695b7cc7-8tsbz\" (UID: \"2621728c-025f-489c-a811-97f005237bc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"
Sep 10 21:11:22.623455 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.623426    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjlh\" (UniqueName: \"kubernetes.io/projected/2621728c-025f-489c-a811-97f005237bc1-kube-api-access-fjjlh\") pod \"volume-data-source-validator-86695b7cc7-8tsbz\" (UID: \"2621728c-025f-489c-a811-97f005237bc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"
Sep 10 21:11:22.722307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.722215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"
Sep 10 21:11:22.989820 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:22.989757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz"]
Sep 10 21:11:22.993344 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:22.993322    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2621728c_025f_489c_a811_97f005237bc1.slice/crio-6861680a865a67a08212016164914afaf9106dc990c806c125b668879282014f WatchSource:0}: Error finding container 6861680a865a67a08212016164914afaf9106dc990c806c125b668879282014f: Status 404 returned error can't find the container with id 6861680a865a67a08212016164914afaf9106dc990c806c125b668879282014f
Sep 10 21:11:23.038939 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:23.038913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz" event={"ID":"2621728c-025f-489c-a811-97f005237bc1","Type":"ContainerStarted","Data":"6861680a865a67a08212016164914afaf9106dc990c806c125b668879282014f"}
Sep 10 21:11:23.974020 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:23.973991    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4rf6f"
Sep 10 21:11:25.045685 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:25.045645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz" event={"ID":"2621728c-025f-489c-a811-97f005237bc1","Type":"ContainerStarted","Data":"f74f9983e2b13f968f9a6667f52e160e6570da386d3c5c855aa1fb6f8608c73b"}
Sep 10 21:11:27.105510 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.105458    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-8tsbz" podStartSLOduration=3.923777114 podStartE2EDuration="5.105442493s" totalImagesPullingTime="1.181665379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:25.060441456 +0000 UTC m=+102.939150517" watchObservedRunningTime="2026-09-10 21:11:27.105442493 +0000 UTC m=+104.984151554"
Sep 10 21:11:27.105930 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.105569    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k2pbk"]
Sep 10 21:11:27.127144 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.127120    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2pbk"]
Sep 10 21:11:27.127260 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.127226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.129612 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.129589    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 21:11:27.129789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.129764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6nt9k\""
Sep 10 21:11:27.129863 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.129804    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 21:11:27.130358 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.130340    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 21:11:27.130453 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.130344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 21:11:27.249572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.249541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-crio-socket\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.249572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.249572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.249746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.249594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbgt\" (UniqueName: \"kubernetes.io/projected/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-api-access-stbgt\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.249746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.249655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-data-volume\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.249746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.249680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350211 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-data-volume\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-crio-socket\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-crio-socket\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350445 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.350399    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.350445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350634 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.350463    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.850446301 +0000 UTC m=+105.729155345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.350634 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.350472    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.350634 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbgt\" (UniqueName: \"kubernetes.io/projected/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-api-access-stbgt\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350634 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.350520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-data-volume\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.350634 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.350535    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.850522697 +0000 UTC m=+105.729231737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.360624 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.360577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbgt\" (UniqueName: \"kubernetes.io/projected/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-api-access-stbgt\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.854532 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.854506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.854687 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:27.854566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:27.854687 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.854645    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.854687 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.854652    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.854788 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.854695    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.854681575 +0000 UTC m=+106.733390616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.854788 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:27.854708    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.854702121 +0000 UTC m=+106.733411160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.174337 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.174307    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"]
Sep 10 21:11:28.176230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.176210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"
Sep 10 21:11:28.178516 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.178499    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 21:11:28.179164 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.179149    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:28.179164 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.179156    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-d2x65\""
Sep 10 21:11:28.186192 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.186171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"]
Sep 10 21:11:28.256913 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.256884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm47m\" (UniqueName: \"kubernetes.io/projected/fcc9febc-b7d0-4695-98a4-aa0d4329d32a-kube-api-access-jm47m\") pod \"migrator-8b4d697c5-l22hq\" (UID: \"fcc9febc-b7d0-4695-98a4-aa0d4329d32a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"
Sep 10 21:11:28.357628 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.357591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm47m\" (UniqueName: \"kubernetes.io/projected/fcc9febc-b7d0-4695-98a4-aa0d4329d32a-kube-api-access-jm47m\") pod \"migrator-8b4d697c5-l22hq\" (UID: \"fcc9febc-b7d0-4695-98a4-aa0d4329d32a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"
Sep 10 21:11:28.365413 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.365384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm47m\" (UniqueName: \"kubernetes.io/projected/fcc9febc-b7d0-4695-98a4-aa0d4329d32a-kube-api-access-jm47m\") pod \"migrator-8b4d697c5-l22hq\" (UID: \"fcc9febc-b7d0-4695-98a4-aa0d4329d32a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"
Sep 10 21:11:28.485310 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.485229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"
Sep 10 21:11:28.750840 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.750772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq"]
Sep 10 21:11:28.753842 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:28.753819    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc9febc_b7d0_4695_98a4_aa0d4329d32a.slice/crio-566a756148176506d0320d7186d34dd5f3f32c3ee727ebabee3157b77cb7e206 WatchSource:0}: Error finding container 566a756148176506d0320d7186d34dd5f3f32c3ee727ebabee3157b77cb7e206: Status 404 returned error can't find the container with id 566a756148176506d0320d7186d34dd5f3f32c3ee727ebabee3157b77cb7e206
Sep 10 21:11:28.821267 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.821246    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5zqd4_a4040f5e-8b82-4b38-ab74-7e50328e2ab0/dns-node-resolver/0.log"
Sep 10 21:11:28.861236 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.861211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:28.861364 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:28.861283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:28.861364 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:28.861346    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.861471 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:28.861394    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.861381642 +0000 UTC m=+108.740090683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.861471 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:28.861391    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.861471 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:28.861452    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.861434934 +0000 UTC m=+108.740143991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:29.055437 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:29.055363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq" event={"ID":"fcc9febc-b7d0-4695-98a4-aa0d4329d32a","Type":"ContainerStarted","Data":"566a756148176506d0320d7186d34dd5f3f32c3ee727ebabee3157b77cb7e206"}
Sep 10 21:11:30.022064 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:30.022039    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fg6nr_a8efe91e-dab3-4f79-882b-3e8e13c07992/node-ca/0.log"
Sep 10 21:11:30.822073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:30.822046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpg2v_714039b9-eb08-4aa6-90c9-260cfb76b60c/serve-healthcheck-canary/0.log"
Sep 10 21:11:30.877281 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:30.877250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:30.877407 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:30.877371    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:30.877447 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:30.877421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:30.877447 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:30.877432    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.877416432 +0000 UTC m=+112.756125487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:30.877553 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:30.877541    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:30.877596 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:30.877587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.877576245 +0000 UTC m=+112.756285329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:31.062570 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:31.062542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq" event={"ID":"fcc9febc-b7d0-4695-98a4-aa0d4329d32a","Type":"ContainerStarted","Data":"944ec0122e6e447f25869165a1b2fad226137f165158f61c007605c12f91bcec"}
Sep 10 21:11:31.062916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:31.062591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq" event={"ID":"fcc9febc-b7d0-4695-98a4-aa0d4329d32a","Type":"ContainerStarted","Data":"c971ac46267b2e6672a1fd3d4ca379ac23baea5da85ec3984ddc861363577505"}
Sep 10 21:11:31.080207 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:31.080132    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-l22hq" podStartSLOduration=1.8015464350000001 podStartE2EDuration="3.080117551s" totalImagesPullingTime="1.278571116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:31.078838302 +0000 UTC m=+108.957547366" watchObservedRunningTime="2026-09-10 21:11:31.080117551 +0000 UTC m=+108.958826612"
Sep 10 21:11:34.903758 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:34.903722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:34.904132 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:34.903779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:34.904132 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:34.903835    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.904132 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:34.903866    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:34.904132 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:34.903905    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.903889475 +0000 UTC m=+120.782598515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.904132 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:34.903921    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.903913621 +0000 UTC m=+120.782622661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.955659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:42.955626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:42.956114 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:42.955678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:42.956114 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:42.955768    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.956114 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:11:42.955818    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls podName:87e9d8a2-47df-4f6a-b538-4c19d54dd1db nodeName:}" failed. No retries permitted until 2026-09-10 21:11:58.955805011 +0000 UTC m=+136.834514065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2pbk" (UID: "87e9d8a2-47df-4f6a-b538-4c19d54dd1db") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.956321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:42.956249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:51.412691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:51.412643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:11:51.414850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:51.414818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef6a4e86-206d-4dba-86cd-b033c338d8b3-metrics-certs\") pod \"network-metrics-daemon-p2vbb\" (UID: \"ef6a4e86-206d-4dba-86cd-b033c338d8b3\") " pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:11:51.420415 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:51.420395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:11:51.428388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:51.428374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2vbb"
Sep 10 21:11:51.695537 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:51.695460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p2vbb"]
Sep 10 21:11:51.699170 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:51.699140    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6a4e86_206d_4dba_86cd_b033c338d8b3.slice/crio-cada0682f441b8f7d016c70a907b364682b7cc902f076229f5d5effd949b8c50 WatchSource:0}: Error finding container cada0682f441b8f7d016c70a907b364682b7cc902f076229f5d5effd949b8c50: Status 404 returned error can't find the container with id cada0682f441b8f7d016c70a907b364682b7cc902f076229f5d5effd949b8c50
Sep 10 21:11:52.119748 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:52.119674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2vbb" event={"ID":"ef6a4e86-206d-4dba-86cd-b033c338d8b3","Type":"ContainerStarted","Data":"cada0682f441b8f7d016c70a907b364682b7cc902f076229f5d5effd949b8c50"}
Sep 10 21:11:53.124660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.124632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2vbb" event={"ID":"ef6a4e86-206d-4dba-86cd-b033c338d8b3","Type":"ContainerStarted","Data":"12aa9613b35ebe5bb9fcf27bd03d76a7fa32d7939537acb9a15480bf11873d1a"}
Sep 10 21:11:53.125059 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.124690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2vbb" event={"ID":"ef6a4e86-206d-4dba-86cd-b033c338d8b3","Type":"ContainerStarted","Data":"6684d8d4ed6f3436e1ebaa347f9679fb48e6b6656ba85a26a0be4c8b6d783aad"}
Sep 10 21:11:53.142762 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.142694    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p2vbb" podStartSLOduration=129.019942226 podStartE2EDuration="2m10.142679186s" totalImagesPullingTime="1.12273696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:53.141827487 +0000 UTC m=+131.020536548" watchObservedRunningTime="2026-09-10 21:11:53.142679186 +0000 UTC m=+131.021388247"
Sep 10 21:11:53.316740 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.316715    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"]
Sep 10 21:11:53.319508 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.319492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.321806 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.321781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 21:11:53.322191 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.322176    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rsf2d\""
Sep 10 21:11:53.322375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.322312    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 21:11:53.322440 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.322407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 21:11:53.324341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.324326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 21:11:53.331530 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.331509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"]
Sep 10 21:11:53.401618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.401551    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"]
Sep 10 21:11:53.404443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.404429    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-wxjs5"]
Sep 10 21:11:53.404560 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.404544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"
Sep 10 21:11:53.407040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.407020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:11:53.407688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.407672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qbq95\""
Sep 10 21:11:53.409062 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.409044    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 21:11:53.409342 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.409325    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-htwvr\""
Sep 10 21:11:53.409430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.409345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 21:11:53.418304 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.418283    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"]
Sep 10 21:11:53.422641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.422623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wxjs5"]
Sep 10 21:11:53.426782 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.426759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1110485c-cf60-48c7-ada7-f418ee32ca4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.426870 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.426844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khdft\" (UniqueName: \"kubernetes.io/projected/1110485c-cf60-48c7-ada7-f418ee32ca4f-kube-api-access-khdft\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.527347 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.527314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tx8d\" (UniqueName: \"kubernetes.io/projected/833ddbd5-6951-4df1-8431-f581407ad1a1-kube-api-access-6tx8d\") pod \"network-check-source-6f8b768455-dsfdk\" (UID: \"833ddbd5-6951-4df1-8431-f581407ad1a1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"
Sep 10 21:11:53.527481 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.527371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khdft\" (UniqueName: \"kubernetes.io/projected/1110485c-cf60-48c7-ada7-f418ee32ca4f-kube-api-access-khdft\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.527481 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.527394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7ctq\" (UniqueName: \"kubernetes.io/projected/b5cd1942-52c2-4dbd-8a8b-e022c1d289a8-kube-api-access-h7ctq\") pod \"downloads-76f99584f4-wxjs5\" (UID: \"b5cd1942-52c2-4dbd-8a8b-e022c1d289a8\") " pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:11:53.527481 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.527449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1110485c-cf60-48c7-ada7-f418ee32ca4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.529877 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.529852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1110485c-cf60-48c7-ada7-f418ee32ca4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.545598 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.545571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khdft\" (UniqueName: \"kubernetes.io/projected/1110485c-cf60-48c7-ada7-f418ee32ca4f-kube-api-access-khdft\") pod \"managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd\" (UID: \"1110485c-cf60-48c7-ada7-f418ee32ca4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.627914 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.627888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tx8d\" (UniqueName: \"kubernetes.io/projected/833ddbd5-6951-4df1-8431-f581407ad1a1-kube-api-access-6tx8d\") pod \"network-check-source-6f8b768455-dsfdk\" (UID: \"833ddbd5-6951-4df1-8431-f581407ad1a1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"
Sep 10 21:11:53.628009 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.627931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7ctq\" (UniqueName: \"kubernetes.io/projected/b5cd1942-52c2-4dbd-8a8b-e022c1d289a8-kube-api-access-h7ctq\") pod \"downloads-76f99584f4-wxjs5\" (UID: \"b5cd1942-52c2-4dbd-8a8b-e022c1d289a8\") " pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:11:53.637388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.637369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"
Sep 10 21:11:53.639943 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.639919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7ctq\" (UniqueName: \"kubernetes.io/projected/b5cd1942-52c2-4dbd-8a8b-e022c1d289a8-kube-api-access-h7ctq\") pod \"downloads-76f99584f4-wxjs5\" (UID: \"b5cd1942-52c2-4dbd-8a8b-e022c1d289a8\") " pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:11:53.640023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.640008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tx8d\" (UniqueName: \"kubernetes.io/projected/833ddbd5-6951-4df1-8431-f581407ad1a1-kube-api-access-6tx8d\") pod \"network-check-source-6f8b768455-dsfdk\" (UID: \"833ddbd5-6951-4df1-8431-f581407ad1a1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"
Sep 10 21:11:53.713667 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.713644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"
Sep 10 21:11:53.718374 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.718353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:11:53.908402 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.908372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd"]
Sep 10 21:11:53.910913 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:53.910881    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1110485c_cf60_48c7_ada7_f418ee32ca4f.slice/crio-9aa24aa4d4b93535642513606895046981c15f5e998560b3fbd998832ff65de4 WatchSource:0}: Error finding container 9aa24aa4d4b93535642513606895046981c15f5e998560b3fbd998832ff65de4: Status 404 returned error can't find the container with id 9aa24aa4d4b93535642513606895046981c15f5e998560b3fbd998832ff65de4
Sep 10 21:11:53.979377 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.979322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk"]
Sep 10 21:11:53.982164 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:53.982139    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833ddbd5_6951_4df1_8431_f581407ad1a1.slice/crio-6e251e1751e87f3cfca024b1d2ae14ea2b738717491407ec5d895bfd679ee22c WatchSource:0}: Error finding container 6e251e1751e87f3cfca024b1d2ae14ea2b738717491407ec5d895bfd679ee22c: Status 404 returned error can't find the container with id 6e251e1751e87f3cfca024b1d2ae14ea2b738717491407ec5d895bfd679ee22c
Sep 10 21:11:53.988045 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:53.987989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wxjs5"]
Sep 10 21:11:53.990843 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:53.990825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cd1942_52c2_4dbd_8a8b_e022c1d289a8.slice/crio-6574aeec91b96ef19b432aefd0fc2233c90030df3ed30f5c724b077201bea31b WatchSource:0}: Error finding container 6574aeec91b96ef19b432aefd0fc2233c90030df3ed30f5c724b077201bea31b: Status 404 returned error can't find the container with id 6574aeec91b96ef19b432aefd0fc2233c90030df3ed30f5c724b077201bea31b
Sep 10 21:11:54.129492 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:54.129461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wxjs5" event={"ID":"b5cd1942-52c2-4dbd-8a8b-e022c1d289a8","Type":"ContainerStarted","Data":"6574aeec91b96ef19b432aefd0fc2233c90030df3ed30f5c724b077201bea31b"}
Sep 10 21:11:54.131017 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:54.130991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk" event={"ID":"833ddbd5-6951-4df1-8431-f581407ad1a1","Type":"ContainerStarted","Data":"45e2a0f6a6a0dd63fe38dabf27b4b2f36e39d8a40d4bf33a48795e6b9ac286cd"}
Sep 10 21:11:54.131140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:54.131037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk" event={"ID":"833ddbd5-6951-4df1-8431-f581407ad1a1","Type":"ContainerStarted","Data":"6e251e1751e87f3cfca024b1d2ae14ea2b738717491407ec5d895bfd679ee22c"}
Sep 10 21:11:54.132172 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:54.132146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd" event={"ID":"1110485c-cf60-48c7-ada7-f418ee32ca4f","Type":"ContainerStarted","Data":"9aa24aa4d4b93535642513606895046981c15f5e998560b3fbd998832ff65de4"}
Sep 10 21:11:54.146469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:54.146431    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-dsfdk" podStartSLOduration=1.146418093 podStartE2EDuration="1.146418093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:54.145327917 +0000 UTC m=+132.024036984" watchObservedRunningTime="2026-09-10 21:11:54.146418093 +0000 UTC m=+132.025127207"
Sep 10 21:11:57.144555 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:57.144523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd" event={"ID":"1110485c-cf60-48c7-ada7-f418ee32ca4f","Type":"ContainerStarted","Data":"baf3914388ce785925dbdda486b9736b8c1f9b466bad2acf39442dd326b806db"}
Sep 10 21:11:57.163228 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:57.163183    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5d9c8476-gmwjd" podStartSLOduration=1.267665269 podStartE2EDuration="4.16316719s" totalImagesPullingTime="2.895501921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:57.162768578 +0000 UTC m=+135.041477640" watchObservedRunningTime="2026-09-10 21:11:57.16316719 +0000 UTC m=+135.041876250"
Sep 10 21:11:58.590970 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.590939    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"]
Sep 10 21:11:58.594371 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.594350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:11:58.596644 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.596619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 21:11:58.596775 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.596752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rnr4g\""
Sep 10 21:11:58.604221 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.603297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"]
Sep 10 21:11:58.672671 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.672635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/79a4b2be-aaef-436d-b171-02a3ce4b8e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9r5sr\" (UID: \"79a4b2be-aaef-436d-b171-02a3ce4b8e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:11:58.773360 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.773324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/79a4b2be-aaef-436d-b171-02a3ce4b8e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9r5sr\" (UID: \"79a4b2be-aaef-436d-b171-02a3ce4b8e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:11:58.775941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.775912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/79a4b2be-aaef-436d-b171-02a3ce4b8e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9r5sr\" (UID: \"79a4b2be-aaef-436d-b171-02a3ce4b8e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:11:58.909104 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.909076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:11:58.976041 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.974809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:58.980358 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:58.978302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87e9d8a2-47df-4f6a-b538-4c19d54dd1db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2pbk\" (UID: \"87e9d8a2-47df-4f6a-b538-4c19d54dd1db\") " pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:59.187376 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:59.187296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"]
Sep 10 21:11:59.190531 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:59.190499    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a4b2be_aaef_436d_b171_02a3ce4b8e67.slice/crio-8ae9abe322a2848e4c83df6424df77d6bd62440b992c36fc075ea9a4686388ff WatchSource:0}: Error finding container 8ae9abe322a2848e4c83df6424df77d6bd62440b992c36fc075ea9a4686388ff: Status 404 returned error can't find the container with id 8ae9abe322a2848e4c83df6424df77d6bd62440b992c36fc075ea9a4686388ff
Sep 10 21:11:59.238230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:59.238201    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6nt9k\""
Sep 10 21:11:59.246885 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:59.246864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2pbk"
Sep 10 21:11:59.530725 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:11:59.530656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2pbk"]
Sep 10 21:11:59.533820 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:11:59.533797    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e9d8a2_47df_4f6a_b538_4c19d54dd1db.slice/crio-796d4d3ed7f26218748b47a8d6f970675987e5967e95fd4c563c157b84774d79 WatchSource:0}: Error finding container 796d4d3ed7f26218748b47a8d6f970675987e5967e95fd4c563c157b84774d79: Status 404 returned error can't find the container with id 796d4d3ed7f26218748b47a8d6f970675987e5967e95fd4c563c157b84774d79
Sep 10 21:12:00.158356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:00.158317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr" event={"ID":"79a4b2be-aaef-436d-b171-02a3ce4b8e67","Type":"ContainerStarted","Data":"8ae9abe322a2848e4c83df6424df77d6bd62440b992c36fc075ea9a4686388ff"}
Sep 10 21:12:00.160428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:00.160330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2pbk" event={"ID":"87e9d8a2-47df-4f6a-b538-4c19d54dd1db","Type":"ContainerStarted","Data":"c467ed6500790b751cb6204993b51043e003570b103188e1c5862a4a786c6c69"}
Sep 10 21:12:00.160428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:00.160390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2pbk" event={"ID":"87e9d8a2-47df-4f6a-b538-4c19d54dd1db","Type":"ContainerStarted","Data":"796d4d3ed7f26218748b47a8d6f970675987e5967e95fd4c563c157b84774d79"}
Sep 10 21:12:01.166216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:01.166183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr" event={"ID":"79a4b2be-aaef-436d-b171-02a3ce4b8e67","Type":"ContainerStarted","Data":"3f427f146964a31e2258063d443ab789e7b61fa7075d6115b40dd976239ecb3a"}
Sep 10 21:12:01.166659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:01.166346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:12:01.168583 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:01.168556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2pbk" event={"ID":"87e9d8a2-47df-4f6a-b538-4c19d54dd1db","Type":"ContainerStarted","Data":"cf72e946a97274c8e2b047370b3df3aae840233b29a024e344768355704b9622"}
Sep 10 21:12:01.171596 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:01.171574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr"
Sep 10 21:12:01.182456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:01.182403    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9r5sr" podStartSLOduration=1.858416812 podStartE2EDuration="3.18238829s" totalImagesPullingTime="1.323971478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:01.181692464 +0000 UTC m=+139.060401529" watchObservedRunningTime="2026-09-10 21:12:01.18238829 +0000 UTC m=+139.061097343"
Sep 10 21:12:03.178867 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.178820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2pbk" event={"ID":"87e9d8a2-47df-4f6a-b538-4c19d54dd1db","Type":"ContainerStarted","Data":"fb8243f2d58520a7d3f159d1c516022d086a323c04e0c402e63ae1cfbb3228f1"}
Sep 10 21:12:03.199698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.199639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k2pbk" podStartSLOduration=33.492402624 podStartE2EDuration="36.199618789s" totalImagesPullingTime="2.707216165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:11:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:03.199300669 +0000 UTC m=+141.078009732" watchObservedRunningTime="2026-09-10 21:12:03.199618789 +0000 UTC m=+141.078327853"
Sep 10 21:12:03.291437 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.291404    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:03.294684 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.294663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.297205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297008    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 21:12:03.297205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297014    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6bhts\""
Sep 10 21:12:03.297205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297092    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 21:12:03.297205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 21:12:03.297509 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 21:12:03.297509 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.297428    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 21:12:03.304836 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.304783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:03.411157 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.411354 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.411354 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.411354 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.411521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.411521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.411407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bj9f\" (UniqueName: \"kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bj9f\" (UniqueName: \"kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512689 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512689 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.512689 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.512533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.513191 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.513162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.513305 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.513206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.513305 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.513217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.515177 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.515151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.515286 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.515163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.520956 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.520933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bj9f\" (UniqueName: \"kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f\") pod \"console-85bf596659-9lfzd\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") " pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.606309 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.606257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:03.888906 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:03.888872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:03.892253 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:03.892215    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f36f8da_d04f_49f0_8031_5fca639d663f.slice/crio-5f381253a6a50881384f014aacee7a32d7d9fe94eef5f5d88dc7cd0f289702b2 WatchSource:0}: Error finding container 5f381253a6a50881384f014aacee7a32d7d9fe94eef5f5d88dc7cd0f289702b2: Status 404 returned error can't find the container with id 5f381253a6a50881384f014aacee7a32d7d9fe94eef5f5d88dc7cd0f289702b2
Sep 10 21:12:04.184655 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:04.184561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf596659-9lfzd" event={"ID":"0f36f8da-d04f-49f0-8031-5fca639d663f","Type":"ContainerStarted","Data":"5f381253a6a50881384f014aacee7a32d7d9fe94eef5f5d88dc7cd0f289702b2"}
Sep 10 21:12:06.057570 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.057526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"]
Sep 10 21:12:06.061133 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.061110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.063416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.063390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 21:12:06.064152 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064126    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:12:06.064284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064125    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:12:06.064284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064217    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cbg7q\""
Sep 10 21:12:06.064368 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064145    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:12:06.064368 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064335    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 21:12:06.064427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.064390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 21:12:06.070803 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.070780    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xpgt8"]
Sep 10 21:12:06.074831 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.074809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.076451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.076430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"]
Sep 10 21:12:06.078044 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.078013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:12:06.078325 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.078299    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:12:06.078545 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.078523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:12:06.078709 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.078692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjpkt\""
Sep 10 21:12:06.237230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.237230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.237476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.237476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-sys\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-textfile\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-root\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-accelerators-collector-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-metrics-client-ca\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-wtmp\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.237905 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbxz8\" (UniqueName: \"kubernetes.io/projected/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-api-access-dbxz8\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.237905 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237905 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blczt\" (UniqueName: \"kubernetes.io/projected/b8482532-6043-4222-a5f3-491e8c032883-kube-api-access-blczt\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.237905 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.237767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339289 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339289 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-sys\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-textfile\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-root\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-accelerators-collector-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-metrics-client-ca\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-wtmp\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxz8\" (UniqueName: \"kubernetes.io/projected/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-api-access-dbxz8\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.339911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.339911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blczt\" (UniqueName: \"kubernetes.io/projected/b8482532-6043-4222-a5f3-491e8c032883-kube-api-access-blczt\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340149 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.339971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.340302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-accelerators-collector-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340421 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.340481 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:12:06.340462    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 21:12:06.340517 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8482532-6043-4222-a5f3-491e8c032883-metrics-client-ca\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340547 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:12:06.340528    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls podName:b8482532-6043-4222-a5f3-491e8c032883 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:06.840510676 +0000 UTC m=+144.719219731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls") pod "node-exporter-xpgt8" (UID: "b8482532-6043-4222-a5f3-491e8c032883") : secret "node-exporter-tls" not found
Sep 10 21:12:06.340631 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-wtmp\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-sys\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8482532-6043-4222-a5f3-491e8c032883-root\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.340978 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.340930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-textfile\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.341296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.341223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.342266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.342240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.342812 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.342790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.343284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.343249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.347666 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.347640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blczt\" (UniqueName: \"kubernetes.io/projected/b8482532-6043-4222-a5f3-491e8c032883-kube-api-access-blczt\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.348601 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.348581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbxz8\" (UniqueName: \"kubernetes.io/projected/adfd4b2e-4a1f-4f5f-8002-d5afccc21606-kube-api-access-dbxz8\") pod \"kube-state-metrics-59fbdd77d-lfcm7\" (UID: \"adfd4b2e-4a1f-4f5f-8002-d5afccc21606\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.373693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.373670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"
Sep 10 21:12:06.843699 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.843664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.846220 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.846186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8482532-6043-4222-a5f3-491e8c032883-node-exporter-tls\") pod \"node-exporter-xpgt8\" (UID: \"b8482532-6043-4222-a5f3-491e8c032883\") " pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:06.989853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:06.989821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xpgt8"
Sep 10 21:12:07.170311 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.170265    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:07.173623 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.173600    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.182208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.182149    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 21:12:07.195640 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.195614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:07.348079 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.348406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.348344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bglkp\" (UniqueName: \"kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.449789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.449789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.449789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bglkp\" (UniqueName: \"kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.449989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450709 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.450683    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450816 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.450749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450816 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.450758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.450922 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.450866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.452519 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.452496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.452627 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.452590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.458624 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.458582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bglkp\" (UniqueName: \"kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp\") pod \"console-69df457867-tfc8d\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") " pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:07.485572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:07.485522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:09.078719 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.078684    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b5d5996d8-69h94"]
Sep 10 21:12:09.082559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.082530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.087547 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9in4i4t4rs5vf\""
Sep 10 21:12:09.087721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-s5gds\""
Sep 10 21:12:09.087721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087595    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 21:12:09.087721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 21:12:09.088004 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087987    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 21:12:09.088004 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.087997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 21:12:09.088103 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.088019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 21:12:09.106437 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.106405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5d5996d8-69h94"]
Sep 10 21:12:09.267328 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-metrics-client-ca\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267492 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267492 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rptrn\" (UniqueName: \"kubernetes.io/projected/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-kube-api-access-rptrn\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267726 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-grpc-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267726 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.267726 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.267715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368715 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368715 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-metrics-client-ca\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rptrn\" (UniqueName: \"kubernetes.io/projected/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-kube-api-access-rptrn\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.368932 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.368927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-grpc-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.369866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.369835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-metrics-client-ca\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372137 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-grpc-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372386 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372429 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.372766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.372745    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-secret-thanos-querier-tls\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.377468 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.377450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rptrn\" (UniqueName: \"kubernetes.io/projected/89ef9e24-aada-4e70-be3c-3d6a44faf1d6-kube-api-access-rptrn\") pod \"thanos-querier-b5d5996d8-69h94\" (UID: \"89ef9e24-aada-4e70-be3c-3d6a44faf1d6\") " pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:09.394327 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:09.394303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:10.378963 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.378935    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"]
Sep 10 21:12:10.382206 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.382184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.385579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.385559    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 21:12:10.385796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.385779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 21:12:10.386392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.386371    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 21:12:10.386488 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.386397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-936n4fnfvrveq\""
Sep 10 21:12:10.386488 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.386375    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 21:12:10.386488 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.386438    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rcmxq\""
Sep 10 21:12:10.393676 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.393653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"]
Sep 10 21:12:10.579624 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-client-certs\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c3c5d3a5-eea9-4013-a696-6026b445195c-audit-log\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-metrics-server-audit-profiles\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-tls\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579972 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-client-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.579972 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.579842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfz7t\" (UniqueName: \"kubernetes.io/projected/c3c5d3a5-eea9-4013-a696-6026b445195c-kube-api-access-pfz7t\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-client-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfz7t\" (UniqueName: \"kubernetes.io/projected/c3c5d3a5-eea9-4013-a696-6026b445195c-kube-api-access-pfz7t\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-client-certs\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c3c5d3a5-eea9-4013-a696-6026b445195c-audit-log\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-metrics-server-audit-profiles\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.680888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.680853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-tls\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.681031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.681005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.681166 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.681145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c3c5d3a5-eea9-4013-a696-6026b445195c-audit-log\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.681636 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.681612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.681857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.681832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c3c5d3a5-eea9-4013-a696-6026b445195c-metrics-server-audit-profiles\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.683262 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.683234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-client-ca-bundle\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.683471 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.683447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-client-certs\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.683825 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.683805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c3c5d3a5-eea9-4013-a696-6026b445195c-secret-metrics-server-tls\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.693839 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.693819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfz7t\" (UniqueName: \"kubernetes.io/projected/c3c5d3a5-eea9-4013-a696-6026b445195c-kube-api-access-pfz7t\") pod \"metrics-server-7b98cbb57b-lp5fx\" (UID: \"c3c5d3a5-eea9-4013-a696-6026b445195c\") " pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:10.861696 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.861667    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:10.993475 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:10.993389    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:12.399192 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.399161    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:12:12.408084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.408057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.410761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.410738    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 21:12:12.410959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.410876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 21:12:12.410959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.410924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 21:12:12.411125 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.410881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 21:12:12.411125 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8adhq63kirtcj\""
Sep 10 21:12:12.411244 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411223    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 21:12:12.411366 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411288    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 21:12:12.411366 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411299    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 21:12:12.411484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 21:12:12.411484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411420    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 21:12:12.411484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 21:12:12.411616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411522    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 21:12:12.411754 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.411735    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 21:12:12.413346 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.413321    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 21:12:12.413786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.413764    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 21:12:12.413786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.413782    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6q9dj\""
Sep 10 21:12:12.420715 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.420696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:12:12.496232 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496232 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcz59\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.496790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.497019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.497019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.497019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.496913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598146 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598146 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.598660 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.599053 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.599053 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598946    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcz59\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.599668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.598641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.602032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.602466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.602425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603030 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.602511    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603030 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.602555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603030 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.602815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.603038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.603062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603419 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.603398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.603855 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.603832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.604287 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.604255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.604654 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.604630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.604742 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.604713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.604860 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.604837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.607887 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.607855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.609741 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.609718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcz59\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59\") pod \"prometheus-k8s-0\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.719013 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.718984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:12.911951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.911917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:12.916016 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.915991    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5d5996d8-69h94"]
Sep 10 21:12:12.918176 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.918141    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"]
Sep 10 21:12:12.918916 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:12.918886    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ef9e24_aada_4e70_be3c_3d6a44faf1d6.slice/crio-729e6b661feb0a25c166e314f04c5534b80d7ec79879b1cf70e86bdc88b9b1db WatchSource:0}: Error finding container 729e6b661feb0a25c166e314f04c5534b80d7ec79879b1cf70e86bdc88b9b1db: Status 404 returned error can't find the container with id 729e6b661feb0a25c166e314f04c5534b80d7ec79879b1cf70e86bdc88b9b1db
Sep 10 21:12:12.922700 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:12.922668    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c5d3a5_eea9_4013_a696_6026b445195c.slice/crio-a77c37da6ef054393a9083205159a45bf33278d620f681bd07870d67f5c3233a WatchSource:0}: Error finding container a77c37da6ef054393a9083205159a45bf33278d620f681bd07870d67f5c3233a: Status 404 returned error can't find the container with id a77c37da6ef054393a9083205159a45bf33278d620f681bd07870d67f5c3233a
Sep 10 21:12:12.943822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:12.943795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7"]
Sep 10 21:12:12.945761 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:12.945737    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadfd4b2e_4a1f_4f5f_8002_d5afccc21606.slice/crio-fbb804ab9fc222c94608ad82132a202d03f772602d47afb20fd57ddc9873575e WatchSource:0}: Error finding container fbb804ab9fc222c94608ad82132a202d03f772602d47afb20fd57ddc9873575e: Status 404 returned error can't find the container with id fbb804ab9fc222c94608ad82132a202d03f772602d47afb20fd57ddc9873575e
Sep 10 21:12:13.025604 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.025511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:12:13.032876 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:13.032840    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e92cfa_e282_4149_a766_23c3fb13aeee.slice/crio-cde5fdd894ac4ed4d99766770afd79f9631e94714558b0edb8284dc0b3e17939 WatchSource:0}: Error finding container cde5fdd894ac4ed4d99766770afd79f9631e94714558b0edb8284dc0b3e17939: Status 404 returned error can't find the container with id cde5fdd894ac4ed4d99766770afd79f9631e94714558b0edb8284dc0b3e17939
Sep 10 21:12:13.227630 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.227223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"cde5fdd894ac4ed4d99766770afd79f9631e94714558b0edb8284dc0b3e17939"}
Sep 10 21:12:13.229124 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.229093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7" event={"ID":"adfd4b2e-4a1f-4f5f-8002-d5afccc21606","Type":"ContainerStarted","Data":"fbb804ab9fc222c94608ad82132a202d03f772602d47afb20fd57ddc9873575e"}
Sep 10 21:12:13.231619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.230988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wxjs5" event={"ID":"b5cd1942-52c2-4dbd-8a8b-e022c1d289a8","Type":"ContainerStarted","Data":"269a4533d6804d576e77e6f978d9825698ec16b9bdddbfa9553090d6d3bc8f08"}
Sep 10 21:12:13.231619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.231307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:12:13.232912 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.232619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx" event={"ID":"c3c5d3a5-eea9-4013-a696-6026b445195c","Type":"ContainerStarted","Data":"a77c37da6ef054393a9083205159a45bf33278d620f681bd07870d67f5c3233a"}
Sep 10 21:12:13.235444 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.235030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69df457867-tfc8d" event={"ID":"7d29861b-c7ac-4cc8-a0c3-34644065b96f","Type":"ContainerStarted","Data":"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"}
Sep 10 21:12:13.235444 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.235075    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69df457867-tfc8d" event={"ID":"7d29861b-c7ac-4cc8-a0c3-34644065b96f","Type":"ContainerStarted","Data":"01209dc526c512974a755649b12a40fdcbe35e0d2ee1873ac57b669ff8693651"}
Sep 10 21:12:13.237250 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.237182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf596659-9lfzd" event={"ID":"0f36f8da-d04f-49f0-8031-5fca639d663f","Type":"ContainerStarted","Data":"2bd9805c0a022ee19af6377109731731abb5380c5ff1f60b5d9a186a26465267"}
Sep 10 21:12:13.238857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.238786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xpgt8" event={"ID":"b8482532-6043-4222-a5f3-491e8c032883","Type":"ContainerStarted","Data":"e175eb01e09860024e4c292bd40f4955c7ee3826a3128a3d8a9886f64f57da17"}
Sep 10 21:12:13.240626 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.240581    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"729e6b661feb0a25c166e314f04c5534b80d7ec79879b1cf70e86bdc88b9b1db"}
Sep 10 21:12:13.241730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.241688    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-wxjs5"
Sep 10 21:12:13.250642 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.250593    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-wxjs5" podStartSLOduration=1.557544155 podStartE2EDuration="20.250575244s" totalImagesPullingTime="18.693031089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:13.249353405 +0000 UTC m=+151.128062470" watchObservedRunningTime="2026-09-10 21:12:13.250575244 +0000 UTC m=+151.129284311"
Sep 10 21:12:13.269458 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.269402    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85bf596659-9lfzd" podStartSLOduration=1.531862963 podStartE2EDuration="10.269383646s" totalImagesPullingTime="8.737520683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:13.267317305 +0000 UTC m=+151.146026369" watchObservedRunningTime="2026-09-10 21:12:13.269383646 +0000 UTC m=+151.148092688"
Sep 10 21:12:13.301742 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.300736    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69df457867-tfc8d" podStartSLOduration=6.300718289 podStartE2EDuration="6.300718289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:13.300683777 +0000 UTC m=+151.179392855" watchObservedRunningTime="2026-09-10 21:12:13.300718289 +0000 UTC m=+151.179427354"
Sep 10 21:12:13.607246 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:13.607169    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:14.253299 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:14.252513    2580 generic.go:356] "Generic (PLEG): container finished" podID="b8482532-6043-4222-a5f3-491e8c032883" containerID="a771b653798667e06107fb416e0b10de3b4e1aba5e735c75adfff7fb88111d3d" exitCode=0
Sep 10 21:12:14.253299 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:14.252560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xpgt8" event={"ID":"b8482532-6043-4222-a5f3-491e8c032883","Type":"ContainerDied","Data":"a771b653798667e06107fb416e0b10de3b4e1aba5e735c75adfff7fb88111d3d"}
Sep 10 21:12:15.260934 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:15.260852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xpgt8" event={"ID":"b8482532-6043-4222-a5f3-491e8c032883","Type":"ContainerStarted","Data":"aa01a6ef9bdef89b77044752c085e87e8e8a3f3fbf56e1b0d48e0b1b1f3ede41"}
Sep 10 21:12:16.113377 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.112906    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65548dd84b-nw87k"]
Sep 10 21:12:16.113377 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:12:16.113330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-65548dd84b-nw87k" podUID="352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"
Sep 10 21:12:16.264016 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.263978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:12:16.292774 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.292750    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:12:16.339649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339625    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc6r4\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339760 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339760 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339760 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339921 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339921 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.339921 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.339869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets\") pod \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\" (UID: \"352ace2f-9c0c-4470-8aec-b9a98e1d9fa2\") "
Sep 10 21:12:16.340071 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.340039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:16.340128 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.340106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:16.340239 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.340194    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-trusted-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.340239 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.340217    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-ca-trust-extracted\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.340386 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.340261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:16.342414 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.342384    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:16.342681 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.342655    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4" (OuterVolumeSpecName: "kube-api-access-bc6r4") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "kube-api-access-bc6r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:16.342681 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.342671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:16.342890 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.342867    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" (UID: "352ace2f-9c0c-4470-8aec-b9a98e1d9fa2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:16.441054 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.441022    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc6r4\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-kube-api-access-bc6r4\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.441054 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.441048    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-image-registry-private-configuration\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.441054 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.441058    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-certificates\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.441295 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.441071    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-bound-sa-token\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:16.441295 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:16.441085    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-installation-pull-secrets\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:17.267651 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.267621    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65548dd84b-nw87k"
Sep 10 21:12:17.306469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.306443    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65548dd84b-nw87k"]
Sep 10 21:12:17.313147 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.313120    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65548dd84b-nw87k"]
Sep 10 21:12:17.347979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.347952    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2-registry-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:17.485707 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.485668    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:17.485881 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.485763    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:17.491382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:17.491352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:18.273925 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.273839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx" event={"ID":"c3c5d3a5-eea9-4013-a696-6026b445195c","Type":"ContainerStarted","Data":"14643b3d33fa47505c00260596d24eb36b2ee44592c760242db1091f38f2b89e"}
Sep 10 21:12:18.276718 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.276657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xpgt8" event={"ID":"b8482532-6043-4222-a5f3-491e8c032883","Type":"ContainerStarted","Data":"6e2016b60f673181ec896dd572a52ddbaf9ce95005b9e9c545b52a5b340faf23"}
Sep 10 21:12:18.279719 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.279690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"0a7e2a18f107d35e2cb0dbe58eff061b4041dc8d0cc3a54ca1917d873b7dc229"}
Sep 10 21:12:18.279902 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.279881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"014956beb9e232d191eae8beec70fa169468a022ae680c3770088deec0453899"}
Sep 10 21:12:18.279984 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.279933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"b281a0e6c3a1aecf464c399e71d80560765deaa2c57f67ec5010d10162929ad8"}
Sep 10 21:12:18.281578 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.281548    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea" exitCode=0
Sep 10 21:12:18.281688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.281630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea"}
Sep 10 21:12:18.284217 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.284191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7" event={"ID":"adfd4b2e-4a1f-4f5f-8002-d5afccc21606","Type":"ContainerStarted","Data":"16d8ce1fa6a50394ec6e75ff4ae624553021441a5b5bba557360ed0fbbf9699d"}
Sep 10 21:12:18.284341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.284253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7" event={"ID":"adfd4b2e-4a1f-4f5f-8002-d5afccc21606","Type":"ContainerStarted","Data":"f54282e1dc15ceaa6e19a0c70e6ce720661868f97f7822a22946c89091b5f836"}
Sep 10 21:12:18.284341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.284320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7" event={"ID":"adfd4b2e-4a1f-4f5f-8002-d5afccc21606","Type":"ContainerStarted","Data":"801094e9a3ad343ab64d08f98671fe6e64e466107285c8ec3f6e2c0eab1bf383"}
Sep 10 21:12:18.289389 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.289368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:18.293936 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.293897    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx" podStartSLOduration=3.701425997 podStartE2EDuration="8.293883542s" totalImagesPullingTime="4.592457545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:18.292457949 +0000 UTC m=+156.171167013" watchObservedRunningTime="2026-09-10 21:12:18.293883542 +0000 UTC m=+156.172592595"
Sep 10 21:12:18.344549 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.344491    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xpgt8" podStartSLOduration=11.272732772 podStartE2EDuration="12.344473981s" totalImagesPullingTime="982.430723ms" totalInitContainerRuntime="89.310486ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:18.344067061 +0000 UTC m=+156.222776125" watchObservedRunningTime="2026-09-10 21:12:18.344473981 +0000 UTC m=+156.223183044"
Sep 10 21:12:18.365395 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.365352    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lfcm7" podStartSLOduration=7.793384523 podStartE2EDuration="12.365336907s" totalImagesPullingTime="4.571952384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:18.363807763 +0000 UTC m=+156.242516825" watchObservedRunningTime="2026-09-10 21:12:18.365336907 +0000 UTC m=+156.244045969"
Sep 10 21:12:18.610611 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:18.610536    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="352ace2f-9c0c-4470-8aec-b9a98e1d9fa2" path="/var/lib/kubelet/pods/352ace2f-9c0c-4470-8aec-b9a98e1d9fa2/volumes"
Sep 10 21:12:19.024629 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:12:19.024580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b8h6t" podUID="550f50e1-7b22-4f80-809a-43044ae513ac"
Sep 10 21:12:20.299749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:20.299573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"2d6244cc3897dc65b079082e84b8fcc1c95d1c2a2cd6b89572b604767251fbbb"}
Sep 10 21:12:20.299749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:20.299655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"31b78323a02413caaffbf2437907f4c9428fa31bc7bd41f8d5f2e5b1902b9109"}
Sep 10 21:12:20.299749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:20.299697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" event={"ID":"89ef9e24-aada-4e70-be3c-3d6a44faf1d6","Type":"ContainerStarted","Data":"eaee6bac64398f94c773b92779f704e4197c86aebd366e04de84e5d984267120"}
Sep 10 21:12:20.300375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:20.299997    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:20.325116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:20.324483    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94" podStartSLOduration=5.385727771 podStartE2EDuration="11.324463862s" totalImagesPullingTime="5.938736091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:20.322817396 +0000 UTC m=+158.201526460" watchObservedRunningTime="2026-09-10 21:12:20.324463862 +0000 UTC m=+158.203172928"
Sep 10 21:12:21.650762 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:21.650137    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:23.316473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.316448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"}
Sep 10 21:12:23.316735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.316516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"}
Sep 10 21:12:23.316735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.316544    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"}
Sep 10 21:12:23.316735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.316568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"}
Sep 10 21:12:23.316735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.316591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"}
Sep 10 21:12:23.910535 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.910503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:23.912773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:23.912749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/550f50e1-7b22-4f80-809a-43044ae513ac-metrics-tls\") pod \"dns-default-b8h6t\" (UID: \"550f50e1-7b22-4f80-809a-43044ae513ac\") " pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:24.324073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:24.323992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerStarted","Data":"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"}
Sep 10 21:12:24.353965 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:24.353913    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.485063567 podStartE2EDuration="12.353895833s" totalImagesPullingTime="8.837242476s" totalInitContainerRuntime="31.58979ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:24.35124783 +0000 UTC m=+162.229956898" watchObservedRunningTime="2026-09-10 21:12:24.353895833 +0000 UTC m=+162.232604894"
Sep 10 21:12:26.310790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:26.310762    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b5d5996d8-69h94"
Sep 10 21:12:27.719995 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:27.719963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:12:30.604842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.604807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:30.607625 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.607605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:12:30.615547 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.615516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:30.932215 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.932186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b8h6t"]
Sep 10 21:12:30.935015 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:12:30.934985    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550f50e1_7b22_4f80_809a_43044ae513ac.slice/crio-74fdeb8ba70073d4da9887cf083a89212eeffa4d8cb4947745a116a404db44fe WatchSource:0}: Error finding container 74fdeb8ba70073d4da9887cf083a89212eeffa4d8cb4947745a116a404db44fe: Status 404 returned error can't find the container with id 74fdeb8ba70073d4da9887cf083a89212eeffa4d8cb4947745a116a404db44fe
Sep 10 21:12:30.994110 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.994085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:30.994199 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:30.994139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:31.354112 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:31.354042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8h6t" event={"ID":"550f50e1-7b22-4f80-809a-43044ae513ac","Type":"ContainerStarted","Data":"74fdeb8ba70073d4da9887cf083a89212eeffa4d8cb4947745a116a404db44fe"}
Sep 10 21:12:33.363418 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:33.363386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8h6t" event={"ID":"550f50e1-7b22-4f80-809a-43044ae513ac","Type":"ContainerStarted","Data":"45eddcf51725097de1f46adae2ccbd91f11ef89edb567bcd86354321f18758f0"}
Sep 10 21:12:33.363794 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:33.363442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8h6t" event={"ID":"550f50e1-7b22-4f80-809a-43044ae513ac","Type":"ContainerStarted","Data":"31a8dd5ab6e93e005fcf7678252d82a3766b2380a0b9d2c47d5e34d25fcf63cc"}
Sep 10 21:12:33.363794 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:33.363540    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:33.382124 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:33.382079    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b8h6t" podStartSLOduration=136.251181298 podStartE2EDuration="2m18.382064296s" totalImagesPullingTime="2.130882998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:33.380730899 +0000 UTC m=+171.259439973" watchObservedRunningTime="2026-09-10 21:12:33.382064296 +0000 UTC m=+171.260773357"
Sep 10 21:12:38.262485 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.262379    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85bf596659-9lfzd" podUID="0f36f8da-d04f-49f0-8031-5fca639d663f" containerName="console" containerID="cri-o://2bd9805c0a022ee19af6377109731731abb5380c5ff1f60b5d9a186a26465267" gracePeriod=15
Sep 10 21:12:38.384855 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.384828    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f36f8da-d04f-49f0-8031-5fca639d663f" containerID="2bd9805c0a022ee19af6377109731731abb5380c5ff1f60b5d9a186a26465267" exitCode=0
Sep 10 21:12:38.384947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.384895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf596659-9lfzd" event={"ID":"0f36f8da-d04f-49f0-8031-5fca639d663f","Type":"ContainerDied","Data":"2bd9805c0a022ee19af6377109731731abb5380c5ff1f60b5d9a186a26465267"}
Sep 10 21:12:38.540951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.540894    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:38.624178 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624316 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624198    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624316 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624224    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624316 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624249    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624316 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624293    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624536 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624331    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bj9f\" (UniqueName: \"kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f\") pod \"0f36f8da-d04f-49f0-8031-5fca639d663f\" (UID: \"0f36f8da-d04f-49f0-8031-5fca639d663f\") "
Sep 10 21:12:38.624662 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config" (OuterVolumeSpecName: "console-config") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:38.624730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca" (OuterVolumeSpecName: "service-ca") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:38.624730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.624663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:38.626417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.626385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:38.626417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.626401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f" (OuterVolumeSpecName: "kube-api-access-2bj9f") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "kube-api-access-2bj9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:38.626543 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.626438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0f36f8da-d04f-49f0-8031-5fca639d663f" (UID: "0f36f8da-d04f-49f0-8031-5fca639d663f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:38.725669 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725643    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-console-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:38.725669 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725665    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:38.725787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725676    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-serving-cert\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:38.725787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725684    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bj9f\" (UniqueName: \"kubernetes.io/projected/0f36f8da-d04f-49f0-8031-5fca639d663f-kube-api-access-2bj9f\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:38.725787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725693    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f36f8da-d04f-49f0-8031-5fca639d663f-console-oauth-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:38.725787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:38.725701    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f36f8da-d04f-49f0-8031-5fca639d663f-oauth-serving-cert\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:39.390142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:39.390105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bf596659-9lfzd" event={"ID":"0f36f8da-d04f-49f0-8031-5fca639d663f","Type":"ContainerDied","Data":"5f381253a6a50881384f014aacee7a32d7d9fe94eef5f5d88dc7cd0f289702b2"}
Sep 10 21:12:39.390142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:39.390132    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bf596659-9lfzd"
Sep 10 21:12:39.390625 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:39.390178    2580 scope.go:117] "RemoveContainer" containerID="2bd9805c0a022ee19af6377109731731abb5380c5ff1f60b5d9a186a26465267"
Sep 10 21:12:39.416006 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:39.415980    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:39.420609 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:39.420588    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85bf596659-9lfzd"]
Sep 10 21:12:40.610092 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:40.610057    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f36f8da-d04f-49f0-8031-5fca639d663f" path="/var/lib/kubelet/pods/0f36f8da-d04f-49f0-8031-5fca639d663f/volumes"
Sep 10 21:12:43.369757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:43.369724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b8h6t"
Sep 10 21:12:46.676940 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:46.676904    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69df457867-tfc8d" podUID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" containerName="console" containerID="cri-o://5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2" gracePeriod=15
Sep 10 21:12:46.966155 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:46.966127    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:47.086112 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086112 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086156    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bglkp\" (UniqueName: \"kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086252    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086447 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086346    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086447 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca\") pod \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\" (UID: \"7d29861b-c7ac-4cc8-a0c3-34644065b96f\") "
Sep 10 21:12:47.086585 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086560    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config" (OuterVolumeSpecName: "console-config") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:47.086585 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:47.086771 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086717    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.086771 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086748    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-trusted-ca-bundle\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.086937 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086835    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca" (OuterVolumeSpecName: "service-ca") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:47.086982 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.086938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:47.088529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.088501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp" (OuterVolumeSpecName: "kube-api-access-bglkp") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "kube-api-access-bglkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:47.088607 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.088578    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:47.088652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.088622    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7d29861b-c7ac-4cc8-a0c3-34644065b96f" (UID: "7d29861b-c7ac-4cc8-a0c3-34644065b96f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:47.187691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.187622    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bglkp\" (UniqueName: \"kubernetes.io/projected/7d29861b-c7ac-4cc8-a0c3-34644065b96f-kube-api-access-bglkp\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.187691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.187646    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-oauth-serving-cert\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.187691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.187656    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-oauth-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.187691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.187665    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d29861b-c7ac-4cc8-a0c3-34644065b96f-console-serving-cert\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.187691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.187674    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d29861b-c7ac-4cc8-a0c3-34644065b96f-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:12:47.423852 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.423810    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" containerID="5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2" exitCode=0
Sep 10 21:12:47.424029 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.423863    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69df457867-tfc8d"
Sep 10 21:12:47.424029 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.423903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69df457867-tfc8d" event={"ID":"7d29861b-c7ac-4cc8-a0c3-34644065b96f","Type":"ContainerDied","Data":"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"}
Sep 10 21:12:47.424029 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.423961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69df457867-tfc8d" event={"ID":"7d29861b-c7ac-4cc8-a0c3-34644065b96f","Type":"ContainerDied","Data":"01209dc526c512974a755649b12a40fdcbe35e0d2ee1873ac57b669ff8693651"}
Sep 10 21:12:47.424029 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.423993    2580 scope.go:117] "RemoveContainer" containerID="5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"
Sep 10 21:12:47.432600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.432584    2580 scope.go:117] "RemoveContainer" containerID="5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"
Sep 10 21:12:47.432895 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:12:47.432878    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2\": container with ID starting with 5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2 not found: ID does not exist" containerID="5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"
Sep 10 21:12:47.432953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.432902    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2"} err="failed to get container status \"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2\": rpc error: code = NotFound desc = could not find container \"5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2\": container with ID starting with 5c60f6e0146e8de0a5ed6ea9a8c6f1976b897607ad11013b25f6011100ab62f2 not found: ID does not exist"
Sep 10 21:12:47.445558 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.445498    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:47.450134 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:47.450112    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69df457867-tfc8d"]
Sep 10 21:12:48.609955 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:48.609917    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" path="/var/lib/kubelet/pods/7d29861b-c7ac-4cc8-a0c3-34644065b96f/volumes"
Sep 10 21:12:50.999453 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:50.999425    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:12:51.003166 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:12:51.003145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b98cbb57b-lp5fx"
Sep 10 21:13:12.719702 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:12.719669    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:12.738614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:12.738591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:13.545813 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:13.545786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:30.847289 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847238    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847573    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy" containerID="cri-o://fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156" gracePeriod=600
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="thanos-sidecar" containerID="cri-o://992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d" gracePeriod=600
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847646    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772" gracePeriod=600
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847659    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-web" containerID="cri-o://1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200" gracePeriod=600
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847635    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="config-reloader" containerID="cri-o://ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739" gracePeriod=600
Sep 10 21:13:30.847811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:30.847559    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="prometheus" containerID="cri-o://4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40" gracePeriod=600
Sep 10 21:13:31.600158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600127    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772" exitCode=0
Sep 10 21:13:31.600158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600149    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156" exitCode=0
Sep 10 21:13:31.600158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600155    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d" exitCode=0
Sep 10 21:13:31.600158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600160    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739" exitCode=0
Sep 10 21:13:31.600158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600165    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40" exitCode=0
Sep 10 21:13:31.600469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"}
Sep 10 21:13:31.600469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"}
Sep 10 21:13:31.600469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"}
Sep 10 21:13:31.600469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"}
Sep 10 21:13:31.600469 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:31.600348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"}
Sep 10 21:13:32.086010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.085986    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.132726 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132702    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcz59\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.132876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132879    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132909    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132939    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.132998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133045    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133163    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133198    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133251    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle\") pod \"20e92cfa-e282-4149-a766-23c3fb13aeee\" (UID: \"20e92cfa-e282-4149-a766-23c3fb13aeee\") "
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133461    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133518    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.133672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133538    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.133980 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.133910    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.134451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.134420    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.134975 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.134878    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.136026 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.135959    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:32.136749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.136711    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.137118 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.137072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:32.137567 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.137522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.137567 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.137514    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59" (OuterVolumeSpecName: "kube-api-access-rcz59") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "kube-api-access-rcz59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:32.137723 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.137686    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.137853 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.137825    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:32.138124 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.138088    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out" (OuterVolumeSpecName: "config-out") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:32.138419 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.138394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.138675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.138639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.138941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.138905    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.139200 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.139180    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.139428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.139411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config" (OuterVolumeSpecName: "config") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.139839 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.139813    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.149691 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.149668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config" (OuterVolumeSpecName: "web-config") pod "20e92cfa-e282-4149-a766-23c3fb13aeee" (UID: "20e92cfa-e282-4149-a766-23c3fb13aeee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:32.234010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.233986    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234006    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234017    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcz59\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-kube-api-access-rcz59\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234027    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234036    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-metrics-client-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234045    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20e92cfa-e282-4149-a766-23c3fb13aeee-tls-assets\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234053    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-kube-rbac-proxy\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234061    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-db\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234068    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234077    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234085    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-grpc-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234093    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234101    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20e92cfa-e282-4149-a766-23c3fb13aeee-config-out\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234108    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-configmap-metrics-client-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234116    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20e92cfa-e282-4149-a766-23c3fb13aeee-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234124    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-web-config\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234133    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234141    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.234615 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.234149    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20e92cfa-e282-4149-a766-23c3fb13aeee-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:13:32.607063 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.606973    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerID="1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200" exitCode=0
Sep 10 21:13:32.607193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.607119    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.610284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.610241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"}
Sep 10 21:13:32.610412 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.610321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20e92cfa-e282-4149-a766-23c3fb13aeee","Type":"ContainerDied","Data":"cde5fdd894ac4ed4d99766770afd79f9631e94714558b0edb8284dc0b3e17939"}
Sep 10 21:13:32.610412 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.610373    2580 scope.go:117] "RemoveContainer" containerID="a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"
Sep 10 21:13:32.619191 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.619173    2580 scope.go:117] "RemoveContainer" containerID="fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"
Sep 10 21:13:32.626041 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.626028    2580 scope.go:117] "RemoveContainer" containerID="1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"
Sep 10 21:13:32.634436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.633414    2580 scope.go:117] "RemoveContainer" containerID="992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"
Sep 10 21:13:32.636676 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.636659    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:32.640755 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.640721    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:32.642722 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.642703    2580 scope.go:117] "RemoveContainer" containerID="ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"
Sep 10 21:13:32.649247 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.649231    2580 scope.go:117] "RemoveContainer" containerID="4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"
Sep 10 21:13:32.655996 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.655978    2580 scope.go:117] "RemoveContainer" containerID="fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea"
Sep 10 21:13:32.662681 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.662660    2580 scope.go:117] "RemoveContainer" containerID="a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"
Sep 10 21:13:32.662910 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.662895    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772\": container with ID starting with a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772 not found: ID does not exist" containerID="a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"
Sep 10 21:13:32.662953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.662918    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772"} err="failed to get container status \"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772\": rpc error: code = NotFound desc = could not find container \"a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772\": container with ID starting with a24410ccd555a0c4b5267ebee72c36d639132bdbd0941017e3ac8d94c1864772 not found: ID does not exist"
Sep 10 21:13:32.662953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.662933    2580 scope.go:117] "RemoveContainer" containerID="fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"
Sep 10 21:13:32.663203 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.663175    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156\": container with ID starting with fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156 not found: ID does not exist" containerID="fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"
Sep 10 21:13:32.663247 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663207    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156"} err="failed to get container status \"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156\": rpc error: code = NotFound desc = could not find container \"fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156\": container with ID starting with fa058205d22e3169ab7ad4e7e2c52958ddd3f92d8e0758883d9b39c283881156 not found: ID does not exist"
Sep 10 21:13:32.663247 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663222    2580 scope.go:117] "RemoveContainer" containerID="1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"
Sep 10 21:13:32.663457 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.663441    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200\": container with ID starting with 1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200 not found: ID does not exist" containerID="1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"
Sep 10 21:13:32.663497 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663460    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200"} err="failed to get container status \"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200\": rpc error: code = NotFound desc = could not find container \"1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200\": container with ID starting with 1a9fecfb2e652175a61e1ea3bf3eadc0e4f19e1ac7060e3c4471a17dc4f0a200 not found: ID does not exist"
Sep 10 21:13:32.663497 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663472    2580 scope.go:117] "RemoveContainer" containerID="992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"
Sep 10 21:13:32.663701 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.663685    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d\": container with ID starting with 992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d not found: ID does not exist" containerID="992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"
Sep 10 21:13:32.663748 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663703    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d"} err="failed to get container status \"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d\": rpc error: code = NotFound desc = could not find container \"992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d\": container with ID starting with 992ad15a90e153ac40c70b8ca1e8fb58e6c95677f323b6223a4d5f958b25bf0d not found: ID does not exist"
Sep 10 21:13:32.663748 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663716    2580 scope.go:117] "RemoveContainer" containerID="ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"
Sep 10 21:13:32.663913 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.663898    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739\": container with ID starting with ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739 not found: ID does not exist" containerID="ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"
Sep 10 21:13:32.663951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663916    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739"} err="failed to get container status \"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739\": rpc error: code = NotFound desc = could not find container \"ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739\": container with ID starting with ce6abc7542bd2ddb7f12b15d409dc67be7c751874672e30e9669deaa909fb739 not found: ID does not exist"
Sep 10 21:13:32.663951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.663928    2580 scope.go:117] "RemoveContainer" containerID="4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"
Sep 10 21:13:32.664140 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.664120    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40\": container with ID starting with 4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40 not found: ID does not exist" containerID="4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"
Sep 10 21:13:32.664193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.664142    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40"} err="failed to get container status \"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40\": rpc error: code = NotFound desc = could not find container \"4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40\": container with ID starting with 4d31cc45e5a26bbebd240c31bf6a31422463a9b999243d9663f8eaabe4db6b40 not found: ID does not exist"
Sep 10 21:13:32.664193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.664153    2580 scope.go:117] "RemoveContainer" containerID="fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea"
Sep 10 21:13:32.664364 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:13:32.664350    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea\": container with ID starting with fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea not found: ID does not exist" containerID="fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea"
Sep 10 21:13:32.664410 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.664366    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea"} err="failed to get container status \"fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea\": rpc error: code = NotFound desc = could not find container \"fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea\": container with ID starting with fa8013c2506598c96d32feb9d14f14fa527b8b416743dac52bcf68213e8936ea not found: ID does not exist"
Sep 10 21:13:32.669730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.669711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:32.669971 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.669959    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy"
Sep 10 21:13:32.670011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.669974    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy"
Sep 10 21:13:32.670011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.669989    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="init-config-reloader"
Sep 10 21:13:32.670011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.669994    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="init-config-reloader"
Sep 10 21:13:32.670011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670003    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="prometheus"
Sep 10 21:13:32.670011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670008    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="prometheus"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670018    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="config-reloader"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670023    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="config-reloader"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670031    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670036    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670045    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="thanos-sidecar"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670051    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="thanos-sidecar"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670056    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f36f8da-d04f-49f0-8031-5fca639d663f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670061    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f36f8da-d04f-49f0-8031-5fca639d663f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670066    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-web"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-web"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670076    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-thanos"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670081    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-thanos"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670121    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670129    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="prometheus"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670135    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f36f8da-d04f-49f0-8031-5fca639d663f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670142    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d29861b-c7ac-4cc8-a0c3-34644065b96f" containerName="console"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670148    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="config-reloader"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670153    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-web"
Sep 10 21:13:32.670150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670157    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="kube-rbac-proxy-thanos"
Sep 10 21:13:32.670665 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.670163    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" containerName="thanos-sidecar"
Sep 10 21:13:32.673657 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.673644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.675908 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.675888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 21:13:32.676002 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.675931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 21:13:32.676002 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.675959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 21:13:32.676162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 21:13:32.676261 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676201    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8adhq63kirtcj\""
Sep 10 21:13:32.676491 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676476    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 21:13:32.676582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676499    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 21:13:32.676702 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 21:13:32.677040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676960    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 21:13:32.677040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.676988    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 21:13:32.677040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.677002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 21:13:32.677040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.677009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6q9dj\""
Sep 10 21:13:32.677260 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.677083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 21:13:32.677496 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.677482    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 21:13:32.678934 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.678915    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 21:13:32.683232 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.683217    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 21:13:32.693434 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.693411    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:32.738159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjx2h\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-kube-api-access-jjx2h\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.738770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.738662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839398 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjx2h\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-kube-api-access-jjx2h\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.839882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.839965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.840135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.840251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840540 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.840322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840540 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.840416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.840944 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.840918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.842594 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.842567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.842685 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.842603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.842761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.842738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.842972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843479 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843534 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.843887 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.843862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.844806 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.844786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.845157 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.845135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.845396 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.845377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.845462 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.845430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.850766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.850747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjx2h\" (UniqueName: \"kubernetes.io/projected/9523f6ee-e4a1-48d2-8acc-0b5073e2b38c-kube-api-access-jjx2h\") pod \"prometheus-k8s-0\" (UID: \"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:32.983635 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:32.983610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:13:33.264836 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:33.264805    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:13:33.266660 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:13:33.266633    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9523f6ee_e4a1_48d2_8acc_0b5073e2b38c.slice/crio-aa1401ae5581c4d5f0c0f234fd63ac50cc9a1d18ff3752a57249adaf0d8dd3b7 WatchSource:0}: Error finding container aa1401ae5581c4d5f0c0f234fd63ac50cc9a1d18ff3752a57249adaf0d8dd3b7: Status 404 returned error can't find the container with id aa1401ae5581c4d5f0c0f234fd63ac50cc9a1d18ff3752a57249adaf0d8dd3b7
Sep 10 21:13:33.612864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:33.612781    2580 generic.go:356] "Generic (PLEG): container finished" podID="9523f6ee-e4a1-48d2-8acc-0b5073e2b38c" containerID="19b50a15265d438d02bc6a56b57b4042cc5ceffd048085839f3e498f1f050578" exitCode=0
Sep 10 21:13:33.612864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:33.612824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerDied","Data":"19b50a15265d438d02bc6a56b57b4042cc5ceffd048085839f3e498f1f050578"}
Sep 10 21:13:33.613072 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:33.612928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"aa1401ae5581c4d5f0c0f234fd63ac50cc9a1d18ff3752a57249adaf0d8dd3b7"}
Sep 10 21:13:34.609766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.609737    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e92cfa-e282-4149-a766-23c3fb13aeee" path="/var/lib/kubelet/pods/20e92cfa-e282-4149-a766-23c3fb13aeee/volumes"
Sep 10 21:13:34.621738 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"861cfefd2ac55f84f3b8df1b75da71cb309942724c35a7456bf82a687fac4c52"}
Sep 10 21:13:34.621842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"432665cbaa10fcaee579da30f85c708c00610a1a880c91e04916cf8cb001d01f"}
Sep 10 21:13:34.621842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"3fa09c6c62fd9c020576996c402dfd9f7baf5a527e83bb9f839a39bb8556da4e"}
Sep 10 21:13:34.621842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"fa4a60307ac98772cd5dcd2a4a804af09d17b24c9236cd58699fd37408ab6111"}
Sep 10 21:13:34.621979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"1d2186660a2ae1460b4ba7b19a892700a2decbb17a5fc44cdcbe35cdf71cadae"}
Sep 10 21:13:34.621979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.621904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9523f6ee-e4a1-48d2-8acc-0b5073e2b38c","Type":"ContainerStarted","Data":"15a82da27e4e166c94b8f649c647907a69a20d48b945c1220b833191139ea863"}
Sep 10 21:13:34.651471 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:34.651426    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.628771563 podStartE2EDuration="2.651412219s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.640656ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:34.649323468 +0000 UTC m=+232.528032531" watchObservedRunningTime="2026-09-10 21:13:34.651412219 +0000 UTC m=+232.530121341"
Sep 10 21:13:37.984510 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:13:37.984475    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:14:32.984656 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:32.984624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:14:32.999851 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:32.999830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:14:33.858951 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:33.858924    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:14:42.520566 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:42.520536    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:14:42.531531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:42.531503    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:14:42.533506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:14:42.533484    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:16:08.033562 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.033531    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7q6vk"]
Sep 10 21:16:08.035821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.035804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.037758 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.037732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4s4f7\""
Sep 10 21:16:08.038299 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.038268    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 21:16:08.038614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.038598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 21:16:08.052132 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.052112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7q6vk"]
Sep 10 21:16:08.078234 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.078209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8msx\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-kube-api-access-h8msx\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.078370 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.078247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-bound-sa-token\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.179315 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.179266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8msx\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-kube-api-access-h8msx\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.179409 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.179326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-bound-sa-token\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.188807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.188773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-bound-sa-token\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.188918 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.188840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8msx\" (UniqueName: \"kubernetes.io/projected/1868d7fa-bf69-4952-a5bf-4a5c01b83b24-kube-api-access-h8msx\") pod \"cert-manager-55b4d954d5-7q6vk\" (UID: \"1868d7fa-bf69-4952-a5bf-4a5c01b83b24\") " pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.344963 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.344888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7q6vk"
Sep 10 21:16:08.612576 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.612507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7q6vk"]
Sep 10 21:16:08.615421 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:16:08.615395    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1868d7fa_bf69_4952_a5bf_4a5c01b83b24.slice/crio-a6e3e478dd61c631d958854bed67cc84ec10694e68a6b7f7dd8fff4697a65bba WatchSource:0}: Error finding container a6e3e478dd61c631d958854bed67cc84ec10694e68a6b7f7dd8fff4697a65bba: Status 404 returned error can't find the container with id a6e3e478dd61c631d958854bed67cc84ec10694e68a6b7f7dd8fff4697a65bba
Sep 10 21:16:08.617233 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:08.617214    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:16:09.198038 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:09.198002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7q6vk" event={"ID":"1868d7fa-bf69-4952-a5bf-4a5c01b83b24","Type":"ContainerStarted","Data":"a6e3e478dd61c631d958854bed67cc84ec10694e68a6b7f7dd8fff4697a65bba"}
Sep 10 21:16:11.931591 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.931558    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"]
Sep 10 21:16:11.933725 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.933707    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:11.936170 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.936147    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jh4rs\""
Sep 10 21:16:11.936536 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.936520    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 21:16:11.936779 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.936765    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:16:11.947889 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:11.947869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"]
Sep 10 21:16:12.012308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.012234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071f660f-e1f3-4219-aee5-28260d566d9e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.012415 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.012302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr87h\" (UniqueName: \"kubernetes.io/projected/071f660f-e1f3-4219-aee5-28260d566d9e-kube-api-access-hr87h\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.113068 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.113040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr87h\" (UniqueName: \"kubernetes.io/projected/071f660f-e1f3-4219-aee5-28260d566d9e-kube-api-access-hr87h\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.113167 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.113086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071f660f-e1f3-4219-aee5-28260d566d9e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.113415 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.113400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071f660f-e1f3-4219-aee5-28260d566d9e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.122460 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.122442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr87h\" (UniqueName: \"kubernetes.io/projected/071f660f-e1f3-4219-aee5-28260d566d9e-kube-api-access-hr87h\") pod \"openshift-lws-operator-fd8ccff4c-np8xn\" (UID: \"071f660f-e1f3-4219-aee5-28260d566d9e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.214202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.214175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7q6vk" event={"ID":"1868d7fa-bf69-4952-a5bf-4a5c01b83b24","Type":"ContainerStarted","Data":"e274e136f6124bcea3f22235ea77b983f1a0c761198fc6653b4e71643bbe490c"}
Sep 10 21:16:12.233205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.233141    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7q6vk" podStartSLOduration=1.10806022 podStartE2EDuration="4.233122424s" totalImagesPullingTime="3.125062204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:12.231933237 +0000 UTC m=+390.110642303" watchObservedRunningTime="2026-09-10 21:16:12.233122424 +0000 UTC m=+390.111831485"
Sep 10 21:16:12.243037 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.243016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"
Sep 10 21:16:12.515065 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:12.515034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn"]
Sep 10 21:16:12.517992 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:16:12.517964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod071f660f_e1f3_4219_aee5_28260d566d9e.slice/crio-59d54ce5a0a43265ecd0cb83bd2d47bdaaed1203923f6bb8e97d32f0c7584b18 WatchSource:0}: Error finding container 59d54ce5a0a43265ecd0cb83bd2d47bdaaed1203923f6bb8e97d32f0c7584b18: Status 404 returned error can't find the container with id 59d54ce5a0a43265ecd0cb83bd2d47bdaaed1203923f6bb8e97d32f0c7584b18
Sep 10 21:16:13.219521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:13.219487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn" event={"ID":"071f660f-e1f3-4219-aee5-28260d566d9e","Type":"ContainerStarted","Data":"59d54ce5a0a43265ecd0cb83bd2d47bdaaed1203923f6bb8e97d32f0c7584b18"}
Sep 10 21:16:16.235009 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:16.234970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn" event={"ID":"071f660f-e1f3-4219-aee5-28260d566d9e","Type":"ContainerStarted","Data":"04a918e15d28041aadb3f9f1046a69187ae80546f48981115ea2dfab985d5503"}
Sep 10 21:16:16.252823 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:16.252776    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-np8xn" podStartSLOduration=2.594838314 podStartE2EDuration="5.252758939s" totalImagesPullingTime="2.657920625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:16.251512912 +0000 UTC m=+394.130221974" watchObservedRunningTime="2026-09-10 21:16:16.252758939 +0000 UTC m=+394.131468001"
Sep 10 21:16:35.619384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.619349    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"]
Sep 10 21:16:35.621528 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.621512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.623817 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.623791    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:16:35.623929 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.623831    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fpp5b\""
Sep 10 21:16:35.624146 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.624132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:16:35.624201 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.624162    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:16:35.624418 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.624406    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:16:35.645547 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.645524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"]
Sep 10 21:16:35.698843 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.698819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.698971 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.698866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.698971 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.698944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.699066 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.698979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.699066 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.699025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hflvn\" (UniqueName: \"kubernetes.io/projected/f734446d-8895-4042-b0c4-0228541dc04b-kube-api-access-hflvn\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.799875 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.799839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.799875 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.799914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.800105 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.799946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.800105 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.799967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hflvn\" (UniqueName: \"kubernetes.io/projected/f734446d-8895-4042-b0c4-0228541dc04b-kube-api-access-hflvn\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.800105 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.800018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.800369 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.800309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.802675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.802629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f734446d-8895-4042-b0c4-0228541dc04b-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.802945 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.802927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.803089 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.803069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f734446d-8895-4042-b0c4-0228541dc04b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.812746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.812705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hflvn\" (UniqueName: \"kubernetes.io/projected/f734446d-8895-4042-b0c4-0228541dc04b-kube-api-access-hflvn\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj\" (UID: \"f734446d-8895-4042-b0c4-0228541dc04b\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:35.932069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:35.932042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:36.215937 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:36.215869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"]
Sep 10 21:16:36.219306 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:16:36.219261    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf734446d_8895_4042_b0c4_0228541dc04b.slice/crio-3bc9b3428ef46290b7477c409b4bcef937a2b77a83901764ce56352ae153d242 WatchSource:0}: Error finding container 3bc9b3428ef46290b7477c409b4bcef937a2b77a83901764ce56352ae153d242: Status 404 returned error can't find the container with id 3bc9b3428ef46290b7477c409b4bcef937a2b77a83901764ce56352ae153d242
Sep 10 21:16:36.312620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:36.312593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj" event={"ID":"f734446d-8895-4042-b0c4-0228541dc04b","Type":"ContainerStarted","Data":"3bc9b3428ef46290b7477c409b4bcef937a2b77a83901764ce56352ae153d242"}
Sep 10 21:16:40.332656 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:40.332617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj" event={"ID":"f734446d-8895-4042-b0c4-0228541dc04b","Type":"ContainerStarted","Data":"631cf492abd5df6a762c45d53c43bd1d6b4a98099c4f5b77ed05a36bfa8d139e"}
Sep 10 21:16:41.338321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:41.338288    2580 generic.go:356] "Generic (PLEG): container finished" podID="f734446d-8895-4042-b0c4-0228541dc04b" containerID="631cf492abd5df6a762c45d53c43bd1d6b4a98099c4f5b77ed05a36bfa8d139e" exitCode=0
Sep 10 21:16:41.338737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:41.338365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj" event={"ID":"f734446d-8895-4042-b0c4-0228541dc04b","Type":"ContainerDied","Data":"631cf492abd5df6a762c45d53c43bd1d6b4a98099c4f5b77ed05a36bfa8d139e"}
Sep 10 21:16:42.344698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:42.344668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj" event={"ID":"f734446d-8895-4042-b0c4-0228541dc04b","Type":"ContainerStarted","Data":"4b75ed60ea03afb57cb0609c6ce5116d01277916bd79c13e7fb5e693610b6b60"}
Sep 10 21:16:42.345059 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:42.344771    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:16:42.366792 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:42.366742    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj" podStartSLOduration=2.69552605 podStartE2EDuration="7.366727374s" totalImagesPullingTime="3.589766107s" totalInitContainerRuntime="1.081435217s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:42.365765913 +0000 UTC m=+420.244474975" watchObservedRunningTime="2026-09-10 21:16:42.366727374 +0000 UTC m=+420.245436437"
Sep 10 21:16:53.351214 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:16:53.351181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj"
Sep 10 21:17:06.450736 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.450636    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wslc9"]
Sep 10 21:17:06.453940 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.453921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.455985 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.455967    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 21:17:06.464086 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.464064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wslc9"]
Sep 10 21:17:06.626345 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.626316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97tbj\" (UniqueName: \"kubernetes.io/projected/a159db2f-b1f3-431b-a52f-623757613c20-kube-api-access-97tbj\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.626345 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.626348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a159db2f-b1f3-431b-a52f-623757613c20-tls-certs\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.727405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.727343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97tbj\" (UniqueName: \"kubernetes.io/projected/a159db2f-b1f3-431b-a52f-623757613c20-kube-api-access-97tbj\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.727405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.727381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a159db2f-b1f3-431b-a52f-623757613c20-tls-certs\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.729713 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.729684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a159db2f-b1f3-431b-a52f-623757613c20-tls-certs\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.735516 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.735491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97tbj\" (UniqueName: \"kubernetes.io/projected/a159db2f-b1f3-431b-a52f-623757613c20-kube-api-access-97tbj\") pod \"model-serving-api-54d89bd5fc-wslc9\" (UID: \"a159db2f-b1f3-431b-a52f-623757613c20\") " pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:06.763829 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:06.763805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:07.031969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:07.031790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wslc9"]
Sep 10 21:17:07.034671 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:17:07.034645    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda159db2f_b1f3_431b_a52f_623757613c20.slice/crio-3dbc2eb83a7146440659f6ad3ad666a6c11114db40261ab6dc78e6f943fa1d92 WatchSource:0}: Error finding container 3dbc2eb83a7146440659f6ad3ad666a6c11114db40261ab6dc78e6f943fa1d92: Status 404 returned error can't find the container with id 3dbc2eb83a7146440659f6ad3ad666a6c11114db40261ab6dc78e6f943fa1d92
Sep 10 21:17:07.458163 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:07.458128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9" event={"ID":"a159db2f-b1f3-431b-a52f-623757613c20","Type":"ContainerStarted","Data":"3dbc2eb83a7146440659f6ad3ad666a6c11114db40261ab6dc78e6f943fa1d92"}
Sep 10 21:17:10.472298 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:10.472249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9" event={"ID":"a159db2f-b1f3-431b-a52f-623757613c20","Type":"ContainerStarted","Data":"8cd5657f0caf97fe1ae704e95be4d3e7369d955c8f1f50f4bf5459f5f38e9ceb"}
Sep 10 21:17:10.472809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:10.472398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:10.490473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:10.490430    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9" podStartSLOduration=1.829897171 podStartE2EDuration="4.490415949s" totalImagesPullingTime="2.660518778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:10.488346264 +0000 UTC m=+448.367055336" watchObservedRunningTime="2026-09-10 21:17:10.490415949 +0000 UTC m=+448.369125011"
Sep 10 21:17:13.924286 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.921116    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"]
Sep 10 21:17:13.995550 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.995520    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"]
Sep 10 21:17:13.995696 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.995605    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:13.998846 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.998813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 21:17:13.998846 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.998835    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c9hw7\""
Sep 10 21:17:13.999056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.998895    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 21:17:13.999056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:13.998895    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 21:17:14.083148 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.083109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.083338 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.083170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-manager-config\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.083338 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.083213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8nwd\" (UniqueName: \"kubernetes.io/projected/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-kube-api-access-f8nwd\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.083338 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.083310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.184638 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.184565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8nwd\" (UniqueName: \"kubernetes.io/projected/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-kube-api-access-f8nwd\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.184638 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.184606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.184809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.184658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.184809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.184687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-manager-config\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.185456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.185433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-manager-config\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.187087 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.187062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-metrics-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.187181 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.187130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-cert\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.195183 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.195161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8nwd\" (UniqueName: \"kubernetes.io/projected/1b1748df-2811-4a3e-a5c7-6e2ba57f198e-kube-api-access-f8nwd\") pod \"lws-controller-manager-66f6f49f76-66srw\" (UID: \"1b1748df-2811-4a3e-a5c7-6e2ba57f198e\") " pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.304266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.304241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:14.589638 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:14.589565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"]
Sep 10 21:17:14.592352 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:17:14.592325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1748df_2811_4a3e_a5c7_6e2ba57f198e.slice/crio-8a4d8b8923af5242d90b4320a2925bde4abd88c393369e6bcdbcf5f5aca7398c WatchSource:0}: Error finding container 8a4d8b8923af5242d90b4320a2925bde4abd88c393369e6bcdbcf5f5aca7398c: Status 404 returned error can't find the container with id 8a4d8b8923af5242d90b4320a2925bde4abd88c393369e6bcdbcf5f5aca7398c
Sep 10 21:17:15.495388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:15.495350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw" event={"ID":"1b1748df-2811-4a3e-a5c7-6e2ba57f198e","Type":"ContainerStarted","Data":"8a4d8b8923af5242d90b4320a2925bde4abd88c393369e6bcdbcf5f5aca7398c"}
Sep 10 21:17:16.502362 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.502328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw" event={"ID":"1b1748df-2811-4a3e-a5c7-6e2ba57f198e","Type":"ContainerStarted","Data":"941db4b492e24a1ddf57a9220860052970cc5027100213f4949822d366e2c91b"}
Sep 10 21:17:16.502757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.502508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:16.530889 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.530838    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw" podStartSLOduration=2.082368356 podStartE2EDuration="3.53081766s" totalImagesPullingTime="1.448449304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:16.527963316 +0000 UTC m=+454.406672379" watchObservedRunningTime="2026-09-10 21:17:16.53081766 +0000 UTC m=+454.409526722"
Sep 10 21:17:16.545031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.545009    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"]
Sep 10 21:17:16.547711 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.547697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.550380 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.550357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mt4fg\""
Sep 10 21:17:16.550689 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.550676    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 21:17:16.567572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.567552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"]
Sep 10 21:17:16.589222 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.589200    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"]
Sep 10 21:17:16.591806 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.591787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.594256 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.594240    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 21:17:16.594789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.594764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nm7rs\""
Sep 10 21:17:16.604413 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.604394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-cert\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.604491 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.604419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpfps\" (UniqueName: \"kubernetes.io/projected/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-kube-api-access-mpfps\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.604593 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.604558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa853146-004c-468b-9cb8-dc5a8e0cc396-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.604716 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.604607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92xlv\" (UniqueName: \"kubernetes.io/projected/fa853146-004c-468b-9cb8-dc5a8e0cc396-kube-api-access-92xlv\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.642779 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.642758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"]
Sep 10 21:17:16.704959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.704932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa853146-004c-468b-9cb8-dc5a8e0cc396-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.705080 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.704966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92xlv\" (UniqueName: \"kubernetes.io/projected/fa853146-004c-468b-9cb8-dc5a8e0cc396-kube-api-access-92xlv\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.705080 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.704987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-cert\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.705258 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.705237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpfps\" (UniqueName: \"kubernetes.io/projected/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-kube-api-access-mpfps\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.707456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.707433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa853146-004c-468b-9cb8-dc5a8e0cc396-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.707549 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.707484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-cert\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.726880 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.726848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92xlv\" (UniqueName: \"kubernetes.io/projected/fa853146-004c-468b-9cb8-dc5a8e0cc396-kube-api-access-92xlv\") pod \"llmisvc-controller-manager-777c6bd7d5-4wj6q\" (UID: \"fa853146-004c-468b-9cb8-dc5a8e0cc396\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:16.726880 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.726876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpfps\" (UniqueName: \"kubernetes.io/projected/eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6-kube-api-access-mpfps\") pod \"kserve-controller-manager-d6f4b65b8-drxqt\" (UID: \"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.857999 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.857933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:16.900718 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:16.900677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:17.142621 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:17.142595    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"]
Sep 10 21:17:17.145656 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:17:17.145634    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda92e62_8e3d_44ed_b5bf_031c7d2aa0c6.slice/crio-1e538f9fd87cb3e11679ca43f862266293c848c04487657aafaaa753b6655f00 WatchSource:0}: Error finding container 1e538f9fd87cb3e11679ca43f862266293c848c04487657aafaaa753b6655f00: Status 404 returned error can't find the container with id 1e538f9fd87cb3e11679ca43f862266293c848c04487657aafaaa753b6655f00
Sep 10 21:17:17.194916 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:17.194895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"]
Sep 10 21:17:17.196690 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:17:17.196666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfa853146_004c_468b_9cb8_dc5a8e0cc396.slice/crio-5d9cd96d5b4493e7ea62066d3ae6451b2e42f55c81f69ca1abdf3b4fdee688a0 WatchSource:0}: Error finding container 5d9cd96d5b4493e7ea62066d3ae6451b2e42f55c81f69ca1abdf3b4fdee688a0: Status 404 returned error can't find the container with id 5d9cd96d5b4493e7ea62066d3ae6451b2e42f55c81f69ca1abdf3b4fdee688a0
Sep 10 21:17:17.508409 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:17.508318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q" event={"ID":"fa853146-004c-468b-9cb8-dc5a8e0cc396","Type":"ContainerStarted","Data":"5d9cd96d5b4493e7ea62066d3ae6451b2e42f55c81f69ca1abdf3b4fdee688a0"}
Sep 10 21:17:17.509620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:17.509588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt" event={"ID":"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6","Type":"ContainerStarted","Data":"1e538f9fd87cb3e11679ca43f862266293c848c04487657aafaaa753b6655f00"}
Sep 10 21:17:20.525351 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:20.525266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt" event={"ID":"eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6","Type":"ContainerStarted","Data":"bdcd0684d2e58e1e17b12da68a336b737794fd0d0d20aa47dd3184f683ad7a17"}
Sep 10 21:17:20.525684 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:20.525394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:17:20.526949 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:20.526930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q" event={"ID":"fa853146-004c-468b-9cb8-dc5a8e0cc396","Type":"ContainerStarted","Data":"e8b91cf77211dd73cbd506bce88f9c4aaba5d5e2d7d9a35b1f797020861d4a4e"}
Sep 10 21:17:20.527060 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:20.527049    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:20.545036 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:20.544993    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt" podStartSLOduration=2.197041587 podStartE2EDuration="4.54497849s" totalImagesPullingTime="2.347936903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:20.543292528 +0000 UTC m=+458.422001591" watchObservedRunningTime="2026-09-10 21:17:20.54497849 +0000 UTC m=+458.423687551"
Sep 10 21:17:21.480889 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:21.480860    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wslc9"
Sep 10 21:17:21.512015 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:21.511964    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q" podStartSLOduration=2.475275367 podStartE2EDuration="5.511949266s" totalImagesPullingTime="3.036673899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:20.559708488 +0000 UTC m=+458.438417552" watchObservedRunningTime="2026-09-10 21:17:21.511949266 +0000 UTC m=+459.390658327"
Sep 10 21:17:27.511848 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:27.511818    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66f6f49f76-66srw"
Sep 10 21:17:51.534097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:51.534064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4wj6q"
Sep 10 21:17:51.536970 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:17:51.536947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-drxqt"
Sep 10 21:18:25.386101 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.386069    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"]
Sep 10 21:18:25.388455 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.388440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.390614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.390592    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x27fp\""
Sep 10 21:18:25.390740 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.390645    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:18:25.390972 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.390955    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:18:25.403314 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.403294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"]
Sep 10 21:18:25.448342 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.448315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfsnn\" (UniqueName: \"kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.448435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.448383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.549408 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.549381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfsnn\" (UniqueName: \"kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.549500 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.549465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.549778 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.549762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.562960 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.562931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfsnn\" (UniqueName: \"kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.698395 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.698335    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:25.974525 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:25.974504    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"]
Sep 10 21:18:25.976894 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:18:25.976867    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e02133_e429_4fa6_8ad1_3983f059de91.slice/crio-ec4662168d23bf62de01bf587c528a7428e4e2b2b463935603a5a1398f35414a WatchSource:0}: Error finding container ec4662168d23bf62de01bf587c528a7428e4e2b2b463935603a5a1398f35414a: Status 404 returned error can't find the container with id ec4662168d23bf62de01bf587c528a7428e4e2b2b463935603a5a1398f35414a
Sep 10 21:18:26.820165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:26.820070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk" event={"ID":"17e02133-e429-4fa6-8ad1-3983f059de91","Type":"ContainerStarted","Data":"ec4662168d23bf62de01bf587c528a7428e4e2b2b463935603a5a1398f35414a"}
Sep 10 21:18:27.978118 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:27.978063    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-84r26"]
Sep 10 21:18:27.985129 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:27.985102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:27.988402 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:27.988300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jl9qb\""
Sep 10 21:18:28.008711 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.008639    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-84r26"]
Sep 10 21:18:28.070246 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.070211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp8cf\" (UniqueName: \"kubernetes.io/projected/707e1800-cdf3-43d7-95c5-d89f92cf3cde-kube-api-access-sp8cf\") pod \"authorino-operator-76b478bd78-84r26\" (UID: \"707e1800-cdf3-43d7-95c5-d89f92cf3cde\") " pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:28.171498 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.171444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp8cf\" (UniqueName: \"kubernetes.io/projected/707e1800-cdf3-43d7-95c5-d89f92cf3cde-kube-api-access-sp8cf\") pod \"authorino-operator-76b478bd78-84r26\" (UID: \"707e1800-cdf3-43d7-95c5-d89f92cf3cde\") " pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:28.182593 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.182551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp8cf\" (UniqueName: \"kubernetes.io/projected/707e1800-cdf3-43d7-95c5-d89f92cf3cde-kube-api-access-sp8cf\") pod \"authorino-operator-76b478bd78-84r26\" (UID: \"707e1800-cdf3-43d7-95c5-d89f92cf3cde\") " pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:28.300320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.300226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:28.658567 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:28.658545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-84r26"]
Sep 10 21:18:29.929750 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:18:29.929704    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707e1800_cdf3_43d7_95c5_d89f92cf3cde.slice/crio-5e7b362f7c207f2f8dec6ae761e7efe04b02c8c29a3762801fc64953ef29cb75 WatchSource:0}: Error finding container 5e7b362f7c207f2f8dec6ae761e7efe04b02c8c29a3762801fc64953ef29cb75: Status 404 returned error can't find the container with id 5e7b362f7c207f2f8dec6ae761e7efe04b02c8c29a3762801fc64953ef29cb75
Sep 10 21:18:30.856859 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:30.856721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk" event={"ID":"17e02133-e429-4fa6-8ad1-3983f059de91","Type":"ContainerStarted","Data":"4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9"}
Sep 10 21:18:30.856859 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:30.856823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:30.860335 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:30.860306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-84r26" event={"ID":"707e1800-cdf3-43d7-95c5-d89f92cf3cde","Type":"ContainerStarted","Data":"5e7b362f7c207f2f8dec6ae761e7efe04b02c8c29a3762801fc64953ef29cb75"}
Sep 10 21:18:30.878582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:30.878177    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk" podStartSLOduration=1.848834896 podStartE2EDuration="5.878152134s" totalImagesPullingTime="4.029317238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:30.876794453 +0000 UTC m=+528.755503516" watchObservedRunningTime="2026-09-10 21:18:30.878152134 +0000 UTC m=+528.756861199"
Sep 10 21:18:31.868205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:31.868170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-84r26" event={"ID":"707e1800-cdf3-43d7-95c5-d89f92cf3cde","Type":"ContainerStarted","Data":"7d098cc63a40049ce78a230ef81631aa1649aca1f27e423ffb4aba64ae76867a"}
Sep 10 21:18:31.868620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:31.868467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:31.913395 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:31.913350    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-84r26" podStartSLOduration=3.54157302 podStartE2EDuration="4.913334703s" totalImagesPullingTime="1.371761683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:31.91135792 +0000 UTC m=+529.790066983" watchObservedRunningTime="2026-09-10 21:18:31.913334703 +0000 UTC m=+529.792043764"
Sep 10 21:18:41.871249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:41.871170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:42.875233 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:42.875208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-84r26"
Sep 10 21:18:43.611058 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.611025    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"]
Sep 10 21:18:43.617708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.617684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.634216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.634187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"]
Sep 10 21:18:43.694432 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.694406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.694532 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.694449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxmn6\" (UniqueName: \"kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.795108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.795074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.795226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.795133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxmn6\" (UniqueName: \"kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.795475 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.795458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.807702 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.807681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxmn6\" (UniqueName: \"kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.927335 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.927311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:43.950936 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.950914    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"]
Sep 10 21:18:43.951146 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.951105    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" containerName="manager" containerID="cri-o://4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9" gracePeriod=2
Sep 10 21:18:43.957170 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.957146    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"]
Sep 10 21:18:43.978816 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.977571    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"]
Sep 10 21:18:43.985208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.985091    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:43.986242 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.985642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" containerName="manager"
Sep 10 21:18:43.986242 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.985664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" containerName="manager"
Sep 10 21:18:43.986242 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.985759    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" containerName="manager"
Sep 10 21:18:43.989509 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.989489    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"]
Sep 10 21:18:43.989637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.989614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:43.992816 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:43.992788    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" is forbidden: User \"system:node:ip-10-0-131-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-178.ec2.internal' and this object" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:43.998870 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:43.998809    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:44.010563 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.010535    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:18:44.015052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.015024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.030507 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.030477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:18:44.030701 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:44.030671    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" is forbidden: User \"system:node:ip-10-0-131-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-178.ec2.internal' and this object" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:44.098761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.098722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.098933 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.098783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w9tz\" (UniqueName: \"kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.098933 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.098815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.098933 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.098892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8fbc\" (UniqueName: \"kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.183626 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.183567    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:44.185833 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:44.185809    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vj8xk\" is forbidden: User \"system:node:ip-10-0-131-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-178.ec2.internal' and this object" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:44.200230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.200352 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w9tz\" (UniqueName: \"kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.200352 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.200352 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8fbc\" (UniqueName: \"kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.200600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.200706 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.200684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.209780 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.209756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8fbc\" (UniqueName: \"kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8vk4k\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.209866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.209756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w9tz\" (UniqueName: \"kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x5hmc\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.301048 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.301021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume\") pod \"17e02133-e429-4fa6-8ad1-3983f059de91\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") "
Sep 10 21:18:44.301179 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.301163    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfsnn\" (UniqueName: \"kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn\") pod \"17e02133-e429-4fa6-8ad1-3983f059de91\" (UID: \"17e02133-e429-4fa6-8ad1-3983f059de91\") "
Sep 10 21:18:44.301429 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.301406    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "17e02133-e429-4fa6-8ad1-3983f059de91" (UID: "17e02133-e429-4fa6-8ad1-3983f059de91"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:44.303060 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.303036    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn" (OuterVolumeSpecName: "kube-api-access-lfsnn") pod "17e02133-e429-4fa6-8ad1-3983f059de91" (UID: "17e02133-e429-4fa6-8ad1-3983f059de91"). InnerVolumeSpecName "kube-api-access-lfsnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:44.331229 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.331206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.338356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.338338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.402311 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.402241    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17e02133-e429-4fa6-8ad1-3983f059de91-extensions-socket-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:44.402430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.402345    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfsnn\" (UniqueName: \"kubernetes.io/projected/17e02133-e429-4fa6-8ad1-3983f059de91-kube-api-access-lfsnn\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:44.610721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.610656    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17e02133-e429-4fa6-8ad1-3983f059de91" path="/var/lib/kubelet/pods/17e02133-e429-4fa6-8ad1-3983f059de91/volumes"
Sep 10 21:18:44.625802 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.625782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:44.627778 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:18:44.627750    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d67f17_86ef_4d08_be01_4da0f6f02562.slice/crio-738815c3bd7660b14e24b281daaa6f77f67aedf8916698ad55b5392c69936875 WatchSource:0}: Error finding container 738815c3bd7660b14e24b281daaa6f77f67aedf8916698ad55b5392c69936875: Status 404 returned error can't find the container with id 738815c3bd7660b14e24b281daaa6f77f67aedf8916698ad55b5392c69936875
Sep 10 21:18:44.636855 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.636837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:18:44.637945 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:18:44.637916    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08a478b_56f6_42a5_9259_86217673d4ab.slice/crio-dec120bec83bf56d53e5b9bb1badc20d9f943dae73f310e0411236a0022a7807 WatchSource:0}: Error finding container dec120bec83bf56d53e5b9bb1badc20d9f943dae73f310e0411236a0022a7807: Status 404 returned error can't find the container with id dec120bec83bf56d53e5b9bb1badc20d9f943dae73f310e0411236a0022a7807
Sep 10 21:18:44.924912 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:44.924883    2580 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 10 21:18:44.924912 ip-10-0-131-178 kubenswrapper[2580]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-pd2l4_kuadrant-system_bc722d47-a180-420b-9f65-f058a9edcc37_0(fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-pd2l4 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb" Netns:"/var/run/netns/685132ff-7275-4afb-8d43-b697200145a6" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-pd2l4;K8S_POD_INFRA_CONTAINER_ID=fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb;K8S_POD_UID=bc722d47-a180-420b-9f65-f058a9edcc37" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4/bc722d47-a180-420b-9f65-f058a9edcc37:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4 fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-pd2l4" not found
Sep 10 21:18:44.924912 ip-10-0-131-178 kubenswrapper[2580]:         '
Sep 10 21:18:44.924912 ip-10-0-131-178 kubenswrapper[2580]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 21:18:44.924912 ip-10-0-131-178 kubenswrapper[2580]:  >
Sep 10 21:18:44.925091 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:44.924936    2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 10 21:18:44.925091 ip-10-0-131-178 kubenswrapper[2580]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-pd2l4_kuadrant-system_bc722d47-a180-420b-9f65-f058a9edcc37_0(fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-pd2l4 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb" Netns:"/var/run/netns/685132ff-7275-4afb-8d43-b697200145a6" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-pd2l4;K8S_POD_INFRA_CONTAINER_ID=fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb;K8S_POD_UID=bc722d47-a180-420b-9f65-f058a9edcc37" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4/bc722d47-a180-420b-9f65-f058a9edcc37:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4 fb6e6ccb7ae751e0713ba63cff3cd771ceb4a505036a97c7d96d20c99573cdeb network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-pd2l4" not found
Sep 10 21:18:44.925091 ip-10-0-131-178 kubenswrapper[2580]:         '
Sep 10 21:18:44.925091 ip-10-0-131-178 kubenswrapper[2580]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 21:18:44.925091 ip-10-0-131-178 kubenswrapper[2580]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:44.929734 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.929694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" event={"ID":"12d67f17-86ef-4d08-be01-4da0f6f02562","Type":"ContainerStarted","Data":"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"}
Sep 10 21:18:44.930113 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.929748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" event={"ID":"12d67f17-86ef-4d08-be01-4da0f6f02562","Type":"ContainerStarted","Data":"738815c3bd7660b14e24b281daaa6f77f67aedf8916698ad55b5392c69936875"}
Sep 10 21:18:44.930113 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.929804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:44.931877 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.931854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" event={"ID":"b08a478b-56f6-42a5-9259-86217673d4ab","Type":"ContainerStarted","Data":"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"}
Sep 10 21:18:44.931992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.931898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" event={"ID":"b08a478b-56f6-42a5-9259-86217673d4ab","Type":"ContainerStarted","Data":"dec120bec83bf56d53e5b9bb1badc20d9f943dae73f310e0411236a0022a7807"}
Sep 10 21:18:44.931992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.931961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:44.933303 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.933265    2580 generic.go:356] "Generic (PLEG): container finished" podID="17e02133-e429-4fa6-8ad1-3983f059de91" containerID="4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9" exitCode=0
Sep 10 21:18:44.933388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.933304    2580 scope.go:117] "RemoveContainer" containerID="4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9"
Sep 10 21:18:44.933388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.933316    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vj8xk"
Sep 10 21:18:44.941346 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.941329    2580 scope.go:117] "RemoveContainer" containerID="4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9"
Sep 10 21:18:44.941602 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:44.941584    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9\": container with ID starting with 4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9 not found: ID does not exist" containerID="4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9"
Sep 10 21:18:44.941650 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.941608    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9"} err="failed to get container status \"4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9\": rpc error: code = NotFound desc = could not find container \"4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9\": container with ID starting with 4499f164af045ef8e55441cbf9c3b71eeb491091cea1293cd2fefcee958b77b9 not found: ID does not exist"
Sep 10 21:18:44.954156 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.954111    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" podStartSLOduration=1.954098849 podStartE2EDuration="1.954098849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:44.952680712 +0000 UTC m=+542.831389775" watchObservedRunningTime="2026-09-10 21:18:44.954098849 +0000 UTC m=+542.832807912"
Sep 10 21:18:44.978177 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:44.978132    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" podStartSLOduration=1.9781180470000002 podStartE2EDuration="1.978118047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:44.976828511 +0000 UTC m=+542.855537574" watchObservedRunningTime="2026-09-10 21:18:44.978118047 +0000 UTC m=+542.856827109"
Sep 10 21:18:45.939587 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:45.939563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:45.947518 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:45.947496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:45.949526 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:45.949506    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" is forbidden: User \"system:node:ip-10-0-131-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-178.ec2.internal' and this object" podUID="bc722d47-a180-420b-9f65-f058a9edcc37" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:46.015640 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.015616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume\") pod \"bc722d47-a180-420b-9f65-f058a9edcc37\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") "
Sep 10 21:18:46.015743 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.015656    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxmn6\" (UniqueName: \"kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6\") pod \"bc722d47-a180-420b-9f65-f058a9edcc37\" (UID: \"bc722d47-a180-420b-9f65-f058a9edcc37\") "
Sep 10 21:18:46.015846 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.015826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bc722d47-a180-420b-9f65-f058a9edcc37" (UID: "bc722d47-a180-420b-9f65-f058a9edcc37"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:46.017695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.017675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6" (OuterVolumeSpecName: "kube-api-access-bxmn6") pod "bc722d47-a180-420b-9f65-f058a9edcc37" (UID: "bc722d47-a180-420b-9f65-f058a9edcc37"). InnerVolumeSpecName "kube-api-access-bxmn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:46.116651 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.116630    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc722d47-a180-420b-9f65-f058a9edcc37-extensions-socket-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:46.116651 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.116649    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxmn6\" (UniqueName: \"kubernetes.io/projected/bc722d47-a180-420b-9f65-f058a9edcc37-kube-api-access-bxmn6\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:46.611340 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.611311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc722d47-a180-420b-9f65-f058a9edcc37" path="/var/lib/kubelet/pods/bc722d47-a180-420b-9f65-f058a9edcc37/volumes"
Sep 10 21:18:46.944216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:46.944188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:46.948339 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:46.948314    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-pd2l4\" is forbidden: User \"system:node:ip-10-0-131-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-178.ec2.internal' and this object" podUID="bc722d47-a180-420b-9f65-f058a9edcc37" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pd2l4"
Sep 10 21:18:55.941380 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:55.941348    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:55.941950 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:55.941415    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:18:56.025428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.025395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:56.025617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.025576    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" podUID="12d67f17-86ef-4d08-be01-4da0f6f02562" containerName="manager" containerID="cri-o://6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617" gracePeriod=10
Sep 10 21:18:56.265515 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.265486    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:56.347712 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.347687    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:18:56.348059 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.348035    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12d67f17-86ef-4d08-be01-4da0f6f02562" containerName="manager"
Sep 10 21:18:56.348059 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.348053    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="12d67f17-86ef-4d08-be01-4da0f6f02562" containerName="manager"
Sep 10 21:18:56.348207 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.348131    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="12d67f17-86ef-4d08-be01-4da0f6f02562" containerName="manager"
Sep 10 21:18:56.350456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.350441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.371553 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.371527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:18:56.394896 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.394877    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume\") pod \"12d67f17-86ef-4d08-be01-4da0f6f02562\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") "
Sep 10 21:18:56.395016 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.394934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8fbc\" (UniqueName: \"kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc\") pod \"12d67f17-86ef-4d08-be01-4da0f6f02562\" (UID: \"12d67f17-86ef-4d08-be01-4da0f6f02562\") "
Sep 10 21:18:56.395341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.395315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12d67f17-86ef-4d08-be01-4da0f6f02562" (UID: "12d67f17-86ef-4d08-be01-4da0f6f02562"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:56.397265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.397243    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc" (OuterVolumeSpecName: "kube-api-access-g8fbc") pod "12d67f17-86ef-4d08-be01-4da0f6f02562" (UID: "12d67f17-86ef-4d08-be01-4da0f6f02562"). InnerVolumeSpecName "kube-api-access-g8fbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:56.495810 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.495783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh4t9\" (UniqueName: \"kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.495928 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.495844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.495969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.495937    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d67f17-86ef-4d08-be01-4da0f6f02562-extensions-socket-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:56.495969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.495955    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8fbc\" (UniqueName: \"kubernetes.io/projected/12d67f17-86ef-4d08-be01-4da0f6f02562-kube-api-access-g8fbc\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:18:56.596573 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.596512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.596674 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.596579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh4t9\" (UniqueName: \"kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.596877 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.596861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.622938 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.622910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh4t9\" (UniqueName: \"kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dmq45\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.659364 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.659333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:56.939328 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.939298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:18:56.941772 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:18:56.941747    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1da87e6_d64f_4cc6_b467_51e6143209df.slice/crio-9b21f12e8d8192f1c4aea5841f7495b083e18b74d379d5c6aef13e87da4d67d5 WatchSource:0}: Error finding container 9b21f12e8d8192f1c4aea5841f7495b083e18b74d379d5c6aef13e87da4d67d5: Status 404 returned error can't find the container with id 9b21f12e8d8192f1c4aea5841f7495b083e18b74d379d5c6aef13e87da4d67d5
Sep 10 21:18:56.994559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.994531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" event={"ID":"d1da87e6-d64f-4cc6-b467-51e6143209df","Type":"ContainerStarted","Data":"9b21f12e8d8192f1c4aea5841f7495b083e18b74d379d5c6aef13e87da4d67d5"}
Sep 10 21:18:56.995901 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.995878    2580 generic.go:356] "Generic (PLEG): container finished" podID="12d67f17-86ef-4d08-be01-4da0f6f02562" containerID="6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617" exitCode=0
Sep 10 21:18:56.996005 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.995916    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"
Sep 10 21:18:56.996005 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.995959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" event={"ID":"12d67f17-86ef-4d08-be01-4da0f6f02562","Type":"ContainerDied","Data":"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"}
Sep 10 21:18:56.996086 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.996009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k" event={"ID":"12d67f17-86ef-4d08-be01-4da0f6f02562","Type":"ContainerDied","Data":"738815c3bd7660b14e24b281daaa6f77f67aedf8916698ad55b5392c69936875"}
Sep 10 21:18:56.996086 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:56.996037    2580 scope.go:117] "RemoveContainer" containerID="6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"
Sep 10 21:18:57.003990 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:57.003924    2580 scope.go:117] "RemoveContainer" containerID="6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"
Sep 10 21:18:57.004239 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:18:57.004217    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617\": container with ID starting with 6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617 not found: ID does not exist" containerID="6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"
Sep 10 21:18:57.004342 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:57.004247    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617"} err="failed to get container status \"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617\": rpc error: code = NotFound desc = could not find container \"6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617\": container with ID starting with 6f59e74f1386a4fb5ad30ef4801fa255424623671b89ac502faafe27dca44617 not found: ID does not exist"
Sep 10 21:18:57.021603 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:57.021583    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:57.030867 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:57.030848    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8vk4k"]
Sep 10 21:18:58.004015 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:58.003978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" event={"ID":"d1da87e6-d64f-4cc6-b467-51e6143209df","Type":"ContainerStarted","Data":"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"}
Sep 10 21:18:58.004492 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:58.004073    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:18:58.028942 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:58.028899    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" podStartSLOduration=2.028885035 podStartE2EDuration="2.028885035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:58.027580945 +0000 UTC m=+555.906290008" watchObservedRunningTime="2026-09-10 21:18:58.028885035 +0000 UTC m=+555.907594096"
Sep 10 21:18:58.611113 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:18:58.611081    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12d67f17-86ef-4d08-be01-4da0f6f02562" path="/var/lib/kubelet/pods/12d67f17-86ef-4d08-be01-4da0f6f02562/volumes"
Sep 10 21:19:09.013893 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.013862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:19:09.079968 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.079931    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:19:09.080219 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.080189    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" podUID="b08a478b-56f6-42a5-9259-86217673d4ab" containerName="manager" containerID="cri-o://f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209" gracePeriod=10
Sep 10 21:19:09.317243 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.317222    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:19:09.397613 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.397576    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume\") pod \"b08a478b-56f6-42a5-9259-86217673d4ab\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") "
Sep 10 21:19:09.397757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.397642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w9tz\" (UniqueName: \"kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz\") pod \"b08a478b-56f6-42a5-9259-86217673d4ab\" (UID: \"b08a478b-56f6-42a5-9259-86217673d4ab\") "
Sep 10 21:19:09.398071 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.398044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b08a478b-56f6-42a5-9259-86217673d4ab" (UID: "b08a478b-56f6-42a5-9259-86217673d4ab"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:09.399892 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.399868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz" (OuterVolumeSpecName: "kube-api-access-2w9tz") pod "b08a478b-56f6-42a5-9259-86217673d4ab" (UID: "b08a478b-56f6-42a5-9259-86217673d4ab"). InnerVolumeSpecName "kube-api-access-2w9tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:09.498463 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.498440    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w9tz\" (UniqueName: \"kubernetes.io/projected/b08a478b-56f6-42a5-9259-86217673d4ab-kube-api-access-2w9tz\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:19:09.498463 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:09.498461    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b08a478b-56f6-42a5-9259-86217673d4ab-extensions-socket-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:19:10.066781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.066748    2580 generic.go:356] "Generic (PLEG): container finished" podID="b08a478b-56f6-42a5-9259-86217673d4ab" containerID="f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209" exitCode=0
Sep 10 21:19:10.067162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.066794    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"
Sep 10 21:19:10.067162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.066837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" event={"ID":"b08a478b-56f6-42a5-9259-86217673d4ab","Type":"ContainerDied","Data":"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"}
Sep 10 21:19:10.067162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.066884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc" event={"ID":"b08a478b-56f6-42a5-9259-86217673d4ab","Type":"ContainerDied","Data":"dec120bec83bf56d53e5b9bb1badc20d9f943dae73f310e0411236a0022a7807"}
Sep 10 21:19:10.067162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.066905    2580 scope.go:117] "RemoveContainer" containerID="f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"
Sep 10 21:19:10.076377 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.076361    2580 scope.go:117] "RemoveContainer" containerID="f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"
Sep 10 21:19:10.076630 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:19:10.076611    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209\": container with ID starting with f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209 not found: ID does not exist" containerID="f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"
Sep 10 21:19:10.076675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.076637    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209"} err="failed to get container status \"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209\": rpc error: code = NotFound desc = could not find container \"f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209\": container with ID starting with f6f83265e9b5cebd18ab597d813c3d996b980cb87de9be02c9397935a4a28209 not found: ID does not exist"
Sep 10 21:19:10.093311 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.093266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:19:10.096872 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.096853    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x5hmc"]
Sep 10 21:19:10.610802 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:10.610773    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b08a478b-56f6-42a5-9259-86217673d4ab" path="/var/lib/kubelet/pods/b08a478b-56f6-42a5-9259-86217673d4ab/volumes"
Sep 10 21:19:42.558087 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:42.558056    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:19:42.559535 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:19:42.559512    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:21:03.609063 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.609028    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"]
Sep 10 21:21:03.609648 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.609400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b08a478b-56f6-42a5-9259-86217673d4ab" containerName="manager"
Sep 10 21:21:03.609648 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.609413    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b08a478b-56f6-42a5-9259-86217673d4ab" containerName="manager"
Sep 10 21:21:03.609648 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.609468    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b08a478b-56f6-42a5-9259-86217673d4ab" containerName="manager"
Sep 10 21:21:03.612454 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.612432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.615871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.615787    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:21:03.615871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.615805    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sprxz\""
Sep 10 21:21:03.615871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.615865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 21:21:03.616097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.615879    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:21:03.624720 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.624702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"]
Sep 10 21:21:03.676506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5rrv\" (UniqueName: \"kubernetes.io/projected/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-kube-api-access-d5rrv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.676601 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.676601 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.676688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.676688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.676751 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.676704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.777899 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.777867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.777921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.777965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.777986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.778020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.778054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5rrv\" (UniqueName: \"kubernetes.io/projected/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-kube-api-access-d5rrv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778324 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.778294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778383 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.778352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.778436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.778391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.780093 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.780077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.780292 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.780253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.791890 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.791870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5rrv\" (UniqueName: \"kubernetes.io/projected/a384ab32-fd70-4c81-b1a4-ecf3e23b9456-kube-api-access-d5rrv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf\" (UID: \"a384ab32-fd70-4c81-b1a4-ecf3e23b9456\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:03.922810 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:03.922780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:04.196147 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:04.196061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"]
Sep 10 21:21:04.200148 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:21:04.200102    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda384ab32_fd70_4c81_b1a4_ecf3e23b9456.slice/crio-80c01cea27ad0f3f66d4953673e4b04b209e2c5e37d679c3c31f3bd9e8bc6968 WatchSource:0}: Error finding container 80c01cea27ad0f3f66d4953673e4b04b209e2c5e37d679c3c31f3bd9e8bc6968: Status 404 returned error can't find the container with id 80c01cea27ad0f3f66d4953673e4b04b209e2c5e37d679c3c31f3bd9e8bc6968
Sep 10 21:21:04.600520 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:04.600435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf" event={"ID":"a384ab32-fd70-4c81-b1a4-ecf3e23b9456","Type":"ContainerStarted","Data":"80c01cea27ad0f3f66d4953673e4b04b209e2c5e37d679c3c31f3bd9e8bc6968"}
Sep 10 21:21:05.614300 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:05.614208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf" event={"ID":"a384ab32-fd70-4c81-b1a4-ecf3e23b9456","Type":"ContainerStarted","Data":"19a1915c9c845356cb86ca655402ead64bd746cc0bd48c4aa5050f362e56410f"}
Sep 10 21:21:05.633122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:05.633074    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf" podStartSLOduration=1.4842667729999999 podStartE2EDuration="2.633059176s" totalImagesPullingTime="1.148792403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:05.632203832 +0000 UTC m=+683.510912896" watchObservedRunningTime="2026-09-10 21:21:05.633059176 +0000 UTC m=+683.511768237"
Sep 10 21:21:13.923295 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:13.923244    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:13.923687 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:13.923310    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:13.935910 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:13.935887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:14.668944 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:14.668912    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf"
Sep 10 21:21:29.710453 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.710409    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"]
Sep 10 21:21:29.714066 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.714046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.716369 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.716352    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 21:21:29.722807 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.722785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"]
Sep 10 21:21:29.789573 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flz4r\" (UniqueName: \"kubernetes.io/projected/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-kube-api-access-flz4r\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.789737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.789737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.789927 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.789927 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.789927 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.789894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flz4r\" (UniqueName: \"kubernetes.io/projected/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-kube-api-access-flz4r\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890465 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890465 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890465 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890465 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890817 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890880 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.890928 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.890859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.892695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.892676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.892986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.892966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:29.897698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:29.897677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flz4r\" (UniqueName: \"kubernetes.io/projected/8d93bf61-7e78-4d99-943c-bdbe7b3c5565-kube-api-access-flz4r\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf\" (UID: \"8d93bf61-7e78-4d99-943c-bdbe7b3c5565\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:30.025992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.025914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:30.296407 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.296318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"]
Sep 10 21:21:30.298607 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:21:30.298580    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d93bf61_7e78_4d99_943c_bdbe7b3c5565.slice/crio-eda0b0c16c4d988d5533371572eb010033e420ec0a9d8ea461593b61df13b118 WatchSource:0}: Error finding container eda0b0c16c4d988d5533371572eb010033e420ec0a9d8ea461593b61df13b118: Status 404 returned error can't find the container with id eda0b0c16c4d988d5533371572eb010033e420ec0a9d8ea461593b61df13b118
Sep 10 21:21:30.299929 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.299909    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:21:30.733304 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.733253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf" event={"ID":"8d93bf61-7e78-4d99-943c-bdbe7b3c5565","Type":"ContainerStarted","Data":"00d0acc5388861591421a8811a9cb9acf8ab691daf1fc28d6bad77e3038eea7b"}
Sep 10 21:21:30.733687 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.733322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf" event={"ID":"8d93bf61-7e78-4d99-943c-bdbe7b3c5565","Type":"ContainerStarted","Data":"eda0b0c16c4d988d5533371572eb010033e420ec0a9d8ea461593b61df13b118"}
Sep 10 21:21:30.750935 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:30.750890    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf" podStartSLOduration=1.545704529 podStartE2EDuration="1.75087585s" totalImagesPullingTime="205.171321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:30.748944423 +0000 UTC m=+708.627653484" watchObservedRunningTime="2026-09-10 21:21:30.75087585 +0000 UTC m=+708.629584912"
Sep 10 21:21:40.026543 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:40.026463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:40.026543 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:40.026519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:40.038737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:40.038713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:40.790248 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:40.790217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf"
Sep 10 21:21:54.017955 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.017913    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"]
Sep 10 21:21:54.021430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.021415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.023825 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.023805    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 21:21:54.032738 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.032718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"]
Sep 10 21:21:54.080871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.080840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.080871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.080872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.081051 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.080901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.081051 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.080915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a725897-272e-4ba1-a4fa-24776df980c6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.081051 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.081008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.081051 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.081045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7dds\" (UniqueName: \"kubernetes.io/projected/5a725897-272e-4ba1-a4fa-24776df980c6-kube-api-access-c7dds\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a725897-272e-4ba1-a4fa-24776df980c6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7dds\" (UniqueName: \"kubernetes.io/projected/5a725897-272e-4ba1-a4fa-24776df980c6-kube-api-access-c7dds\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.181857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.181772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.182108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.182086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.182158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.182111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.182213 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.182198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.184017 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.183992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a725897-272e-4ba1-a4fa-24776df980c6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.184136 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.184061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a725897-272e-4ba1-a4fa-24776df980c6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.189835 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.189809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7dds\" (UniqueName: \"kubernetes.io/projected/5a725897-272e-4ba1-a4fa-24776df980c6-kube-api-access-c7dds\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nxjd7\" (UID: \"5a725897-272e-4ba1-a4fa-24776df980c6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.332866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.332805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:21:54.608235 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:21:54.608203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a725897_272e_4ba1_a4fa_24776df980c6.slice/crio-05a519f2f818b391440f96040c388064a78cad5624baac3602e572cef03e5274 WatchSource:0}: Error finding container 05a519f2f818b391440f96040c388064a78cad5624baac3602e572cef03e5274: Status 404 returned error can't find the container with id 05a519f2f818b391440f96040c388064a78cad5624baac3602e572cef03e5274
Sep 10 21:21:54.612015 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.611995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"]
Sep 10 21:21:54.847701 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:54.847674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7" event={"ID":"5a725897-272e-4ba1-a4fa-24776df980c6","Type":"ContainerStarted","Data":"05a519f2f818b391440f96040c388064a78cad5624baac3602e572cef03e5274"}
Sep 10 21:21:55.855080 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:55.855044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7" event={"ID":"5a725897-272e-4ba1-a4fa-24776df980c6","Type":"ContainerStarted","Data":"54b6326e0f2279c00a8c01c5408fcebb365f5756321d1b2dcc96f05d51666ed1"}
Sep 10 21:21:55.874824 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:55.874778    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7" podStartSLOduration=2.6626053929999998 podStartE2EDuration="2.874762957s" totalImagesPullingTime="212.157564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:55.874112781 +0000 UTC m=+733.752821845" watchObservedRunningTime="2026-09-10 21:21:55.874762957 +0000 UTC m=+733.753472019"
Sep 10 21:21:58.517416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.517388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"]
Sep 10 21:21:58.520935 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.520915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.523170 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.523148    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 21:21:58.536798 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.536775    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"]
Sep 10 21:21:58.617152 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.617341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.617341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a11080-869c-4136-997e-8c948e510796-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.617462 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.617532 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5slz\" (UniqueName: \"kubernetes.io/projected/c4a11080-869c-4136-997e-8c948e510796-kube-api-access-b5slz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.617571 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.617555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.718704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.718850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.718850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a11080-869c-4136-997e-8c948e510796-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.718850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.718850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5slz\" (UniqueName: \"kubernetes.io/projected/c4a11080-869c-4136-997e-8c948e510796-kube-api-access-b5slz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.719049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.718866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.719148 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.719117    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.719291 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.719172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.719291 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.719182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.720993 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.720974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a11080-869c-4136-997e-8c948e510796-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.721318 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.721303    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a11080-869c-4136-997e-8c948e510796-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.728201 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.728182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5slz\" (UniqueName: \"kubernetes.io/projected/c4a11080-869c-4136-997e-8c948e510796-kube-api-access-b5slz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s\" (UID: \"c4a11080-869c-4136-997e-8c948e510796\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:58.830428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:58.830366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:21:59.108313 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:59.108216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"]
Sep 10 21:21:59.111229 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:21:59.111200    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4a11080_869c_4136_997e_8c948e510796.slice/crio-3f58a92c3a69197806d3a63fc435e3c5db5e090f93557583ad4af290a8dd8edd WatchSource:0}: Error finding container 3f58a92c3a69197806d3a63fc435e3c5db5e090f93557583ad4af290a8dd8edd: Status 404 returned error can't find the container with id 3f58a92c3a69197806d3a63fc435e3c5db5e090f93557583ad4af290a8dd8edd
Sep 10 21:21:59.877744 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:59.877709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s" event={"ID":"c4a11080-869c-4136-997e-8c948e510796","Type":"ContainerStarted","Data":"1a041e253275cfbce3bbffdd39845e53df575fb6b0c3e6930aa8f9ced1e43e62"}
Sep 10 21:21:59.877744 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:59.877747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s" event={"ID":"c4a11080-869c-4136-997e-8c948e510796","Type":"ContainerStarted","Data":"3f58a92c3a69197806d3a63fc435e3c5db5e090f93557583ad4af290a8dd8edd"}
Sep 10 21:21:59.895579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:21:59.895538    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s" podStartSLOduration=1.68631748 podStartE2EDuration="1.895518725s" totalImagesPullingTime="209.201245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:59.894883545 +0000 UTC m=+737.773592608" watchObservedRunningTime="2026-09-10 21:21:59.895518725 +0000 UTC m=+737.774227786"
Sep 10 21:22:04.333543 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:04.333508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:22:04.333915 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:04.333571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:22:04.345864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:04.345837    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:22:04.912516 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:04.912488    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nxjd7"
Sep 10 21:22:08.830753 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:08.830722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:22:08.831160 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:08.830779    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:22:08.843902 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:08.843880    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:22:08.931486 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:22:08.931456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s"
Sep 10 21:23:26.002701 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.002672    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:23:26.006416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.006396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.008746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.008720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-780a0cdd-serving-cert\""
Sep 10 21:23:26.008857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.008772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:23:26.009537 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.009516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:23:26.009631 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.009544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:23:26.009631 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.009516    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:23:26.016755 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.016733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:23:26.148702 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.148672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.148864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.148711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.148864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.148792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.148864 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.148808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xc95\" (UniqueName: \"kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.249612 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.249579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.249777 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.249628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.249777 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.249699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.249777 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.249722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xc95\" (UniqueName: \"kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.250344 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.250320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.252162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.252140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.252351 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.252330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.257775 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.257716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xc95\" (UniqueName: \"kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95\") pod \"maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.318599 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.318581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:26.591497 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.591421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:23:26.594242 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:23:26.594210    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bd763b_9899_4063_ab9f_4c256dad1c5b.slice/crio-2cec6ba815d80aeb3f356c18eb8ffa1ce99539884b846254d6b7cf55d5a70f74 WatchSource:0}: Error finding container 2cec6ba815d80aeb3f356c18eb8ffa1ce99539884b846254d6b7cf55d5a70f74: Status 404 returned error can't find the container with id 2cec6ba815d80aeb3f356c18eb8ffa1ce99539884b846254d6b7cf55d5a70f74
Sep 10 21:23:26.815746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.815717    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:23:26.824999 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.824975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:26.827078 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.827058    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-780a0cdd\""
Sep 10 21:23:26.827199 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.827145    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:23:26.827487 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.827467    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:23:26.827975 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.827722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-780a0cdd-dockercfg-kwl5z\""
Sep 10 21:23:26.830475 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.830454    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:23:26.956193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.956161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:26.956363 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:26.956220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lgpf\" (UniqueName: \"kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.057209 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.057175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lgpf\" (UniqueName: \"kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.057540 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.057305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.057907 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.057890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.065359 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.065331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lgpf\" (UniqueName: \"kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf\") pod \"payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") " pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.111440 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.111407    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:23:27.115284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.115256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.117362 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.117327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-f135f4-serving-cert\""
Sep 10 21:23:27.128822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.128800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:23:27.136617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.136593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:27.260548 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.259215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.260548 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.259258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.260548 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.259414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7pph\" (UniqueName: \"kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.260548 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.259451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.291952 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.291889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" event={"ID":"b8bd763b-9899-4063-ab9f-4c256dad1c5b","Type":"ContainerStarted","Data":"2cec6ba815d80aeb3f356c18eb8ffa1ce99539884b846254d6b7cf55d5a70f74"}
Sep 10 21:23:27.360308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.360103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.360308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.360134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.360308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.360182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7pph\" (UniqueName: \"kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.360308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.360204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.360308 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:23:27.360252    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-serving-cert: secret "maas-api-e2e-autores-f135f4-serving-cert" not found
Sep 10 21:23:27.360663 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:23:27.360353    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls podName:c7bc986c-b502-48a9-ba30-4947a0108fc2 nodeName:}" failed. No retries permitted until 2026-09-10 21:23:27.860329527 +0000 UTC m=+825.739038584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls") pod "maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" (UID: "c7bc986c-b502-48a9-ba30-4947a0108fc2") : secret "maas-api-e2e-autores-f135f4-serving-cert" not found
Sep 10 21:23:27.360917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.360872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.363850 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.363827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.369011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.368612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7pph\" (UniqueName: \"kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.419664 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.419533    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:23:27.866928 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.866829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:27.869618 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:27.869511    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") pod \"maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:28.029232 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:28.029197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:28.300097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:28.300060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" event={"ID":"69a44edc-9973-43b2-94b2-2170164f0987","Type":"ContainerStarted","Data":"f3c469058833fdebd06ed2e26ae05336f993f58743b555152fb0888ee9eb3cf5"}
Sep 10 21:23:29.778355 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:29.778311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:23:29.781185 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:23:29.781155    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7bc986c_b502_48a9_ba30_4947a0108fc2.slice/crio-e4b0341967c680a76d6967415ea2a3616c29ea26e14c6fb8d0c891e2d30dcd34 WatchSource:0}: Error finding container e4b0341967c680a76d6967415ea2a3616c29ea26e14c6fb8d0c891e2d30dcd34: Status 404 returned error can't find the container with id e4b0341967c680a76d6967415ea2a3616c29ea26e14c6fb8d0c891e2d30dcd34
Sep 10 21:23:30.312043 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.311955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" event={"ID":"69a44edc-9973-43b2-94b2-2170164f0987","Type":"ContainerStarted","Data":"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"}
Sep 10 21:23:30.312186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.312039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:30.313577 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.313555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" event={"ID":"b8bd763b-9899-4063-ab9f-4c256dad1c5b","Type":"ContainerStarted","Data":"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"}
Sep 10 21:23:30.313720 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.313597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:30.315073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.315048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" event={"ID":"c7bc986c-b502-48a9-ba30-4947a0108fc2","Type":"ContainerStarted","Data":"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"}
Sep 10 21:23:30.315185 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.315088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" event={"ID":"c7bc986c-b502-48a9-ba30-4947a0108fc2","Type":"ContainerStarted","Data":"e4b0341967c680a76d6967415ea2a3616c29ea26e14c6fb8d0c891e2d30dcd34"}
Sep 10 21:23:30.315259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.315187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:30.329753 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.329710    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" podStartSLOduration=2.234447661 podStartE2EDuration="4.32969776s" totalImagesPullingTime="2.095250099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:30.327172603 +0000 UTC m=+828.205881664" watchObservedRunningTime="2026-09-10 21:23:30.32969776 +0000 UTC m=+828.208406824"
Sep 10 21:23:30.353127 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.353088    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" podStartSLOduration=2.434361578 podStartE2EDuration="5.353075673s" totalImagesPullingTime="2.918714095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:30.351972851 +0000 UTC m=+828.230681914" watchObservedRunningTime="2026-09-10 21:23:30.353075673 +0000 UTC m=+828.231784735"
Sep 10 21:23:30.369137 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:30.369072    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" podStartSLOduration=3.106132704 podStartE2EDuration="3.36905972s" totalImagesPullingTime="262.927016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:30.367603353 +0000 UTC m=+828.246312419" watchObservedRunningTime="2026-09-10 21:23:30.36905972 +0000 UTC m=+828.247768781"
Sep 10 21:23:36.324934 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:36.324907    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:23:36.325410 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:36.325306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:23:41.322316 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:41.322267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:23:44.525653 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.525618    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:23:44.529204 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.529185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.531416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.531396    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a6441b-dockercfg-gk4kt\""
Sep 10 21:23:44.531499 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.531396    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a6441b\""
Sep 10 21:23:44.537478 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.537458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:23:44.610249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.610209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.610443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.610423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfqks\" (UniqueName: \"kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.711769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.711731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfqks\" (UniqueName: \"kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.711769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.711774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.712506 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.712487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.719564 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.719541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfqks\" (UniqueName: \"kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks\") pod \"payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:44.840800 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:44.840743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:45.114982 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:45.114957    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:23:45.117566 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:23:45.117542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb17544_c1e7_4860_9fc5_993e3a222958.slice/crio-1239828ec61c47b7b49c31e9e0b986900b84d6821203b42c5a655034a0221b61 WatchSource:0}: Error finding container 1239828ec61c47b7b49c31e9e0b986900b84d6821203b42c5a655034a0221b61: Status 404 returned error can't find the container with id 1239828ec61c47b7b49c31e9e0b986900b84d6821203b42c5a655034a0221b61
Sep 10 21:23:45.396075 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:45.396035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" event={"ID":"abb17544-c1e7-4860-9fc5-993e3a222958","Type":"ContainerStarted","Data":"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"}
Sep 10 21:23:45.396238 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:45.396102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:45.396238 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:45.396122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" event={"ID":"abb17544-c1e7-4860-9fc5-993e3a222958","Type":"ContainerStarted","Data":"1239828ec61c47b7b49c31e9e0b986900b84d6821203b42c5a655034a0221b61"}
Sep 10 21:23:45.412888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:45.412838    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" podStartSLOduration=1.4128229970000001 podStartE2EDuration="1.412822997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:45.412153996 +0000 UTC m=+843.290863055" watchObservedRunningTime="2026-09-10 21:23:45.412822997 +0000 UTC m=+843.291532049"
Sep 10 21:23:56.402572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:56.402538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:23:57.110708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.110675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:23:57.114250 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.114235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.116236 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.116216    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-788af409-kserve-self-signed-certs\""
Sep 10 21:23:57.122503 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.122479    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:23:57.220623 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.220623 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.220834 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.220834 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzmx\" (UniqueName: \"kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.220834 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.220834 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.220802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321918 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.321918 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzmx\" (UniqueName: \"kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.322025 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.321998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.322075 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.322032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.322129 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.322101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.323957 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.323934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.324221 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.324203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.329721 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.329698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzmx\" (UniqueName: \"kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx\") pod \"e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") " pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.423903 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.423877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:23:57.698510 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:57.698485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:23:57.701130 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:23:57.701101    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30d0516f_9e29_4487_8b2b_b01481cfa3c7.slice/crio-c89c998438d738e13394ba5ed09abf4ba627c6b756e0b3279aee4c12c968e727 WatchSource:0}: Error finding container c89c998438d738e13394ba5ed09abf4ba627c6b756e0b3279aee4c12c968e727: Status 404 returned error can't find the container with id c89c998438d738e13394ba5ed09abf4ba627c6b756e0b3279aee4c12c968e727
Sep 10 21:23:58.466358 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:58.466323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" event={"ID":"30d0516f-9e29-4487-8b2b-b01481cfa3c7","Type":"ContainerStarted","Data":"e9daddaf9ee8aef3e34e2cabefecb9b8c87f6ae5e8c27b7576c6ec240815ef75"}
Sep 10 21:23:58.466358 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:58.466361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" event={"ID":"30d0516f-9e29-4487-8b2b-b01481cfa3c7","Type":"ContainerStarted","Data":"c89c998438d738e13394ba5ed09abf4ba627c6b756e0b3279aee4c12c968e727"}
Sep 10 21:23:58.483198 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:23:58.483145    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" podStartSLOduration=1.4831252990000001 podStartE2EDuration="1.483125299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:58.482778825 +0000 UTC m=+856.361487888" watchObservedRunningTime="2026-09-10 21:23:58.483125299 +0000 UTC m=+856.361834362"
Sep 10 21:24:06.826367 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:06.826331    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:24:06.826777 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:06.826540    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" podUID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" containerName="maas-api" containerID="cri-o://ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025" gracePeriod=30
Sep 10 21:24:06.888082 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:06.888044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:24:06.888336 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:06.888309    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" podUID="69a44edc-9973-43b2-94b2-2170164f0987" containerName="payload-processing" containerID="cri-o://1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58" gracePeriod=30
Sep 10 21:24:07.145197 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.145174    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:24:07.162713 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.162638    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:24:07.210216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs\") pod \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") "
Sep 10 21:24:07.210216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca\") pod \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") "
Sep 10 21:24:07.210451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210247    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xc95\" (UniqueName: \"kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95\") pod \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") "
Sep 10 21:24:07.210451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lgpf\" (UniqueName: \"kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf\") pod \"69a44edc-9973-43b2-94b2-2170164f0987\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") "
Sep 10 21:24:07.210451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls\") pod \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\" (UID: \"b8bd763b-9899-4063-ab9f-4c256dad1c5b\") "
Sep 10 21:24:07.210451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume\") pod \"69a44edc-9973-43b2-94b2-2170164f0987\" (UID: \"69a44edc-9973-43b2-94b2-2170164f0987\") "
Sep 10 21:24:07.210920 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210662    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8bd763b-9899-4063-ab9f-4c256dad1c5b" (UID: "b8bd763b-9899-4063-ab9f-4c256dad1c5b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:07.210920 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.210803    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bd763b-9899-4063-ab9f-4c256dad1c5b-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.211135 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.211110    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69a44edc-9973-43b2-94b2-2170164f0987" (UID: "69a44edc-9973-43b2-94b2-2170164f0987"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:07.213191 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.213166    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b8bd763b-9899-4063-ab9f-4c256dad1c5b" (UID: "b8bd763b-9899-4063-ab9f-4c256dad1c5b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:07.213324 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.213186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95" (OuterVolumeSpecName: "kube-api-access-8xc95") pod "b8bd763b-9899-4063-ab9f-4c256dad1c5b" (UID: "b8bd763b-9899-4063-ab9f-4c256dad1c5b"). InnerVolumeSpecName "kube-api-access-8xc95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:07.213553 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.213510    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf" (OuterVolumeSpecName: "kube-api-access-9lgpf") pod "69a44edc-9973-43b2-94b2-2170164f0987" (UID: "69a44edc-9973-43b2-94b2-2170164f0987"). InnerVolumeSpecName "kube-api-access-9lgpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:07.215477 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.215450    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8bd763b-9899-4063-ab9f-4c256dad1c5b" (UID: "b8bd763b-9899-4063-ab9f-4c256dad1c5b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:07.311688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.311658    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.311688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.311683    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xc95\" (UniqueName: \"kubernetes.io/projected/b8bd763b-9899-4063-ab9f-4c256dad1c5b-kube-api-access-8xc95\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.311688 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.311696    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lgpf\" (UniqueName: \"kubernetes.io/projected/69a44edc-9973-43b2-94b2-2170164f0987-kube-api-access-9lgpf\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.311893 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.311705    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8bd763b-9899-4063-ab9f-4c256dad1c5b-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.311893 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.311713    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a44edc-9973-43b2-94b2-2170164f0987-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.425002 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.424974    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:24:07.425168 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.425014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:24:07.437241 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.437222    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:24:07.513320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.513295    2580 generic.go:356] "Generic (PLEG): container finished" podID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" containerID="ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025" exitCode=0
Sep 10 21:24:07.513473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.513394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" event={"ID":"b8bd763b-9899-4063-ab9f-4c256dad1c5b","Type":"ContainerDied","Data":"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"}
Sep 10 21:24:07.513473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.513402    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"
Sep 10 21:24:07.513473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.513434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj" event={"ID":"b8bd763b-9899-4063-ab9f-4c256dad1c5b","Type":"ContainerDied","Data":"2cec6ba815d80aeb3f356c18eb8ffa1ce99539884b846254d6b7cf55d5a70f74"}
Sep 10 21:24:07.513473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.513465    2580 scope.go:117] "RemoveContainer" containerID="ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"
Sep 10 21:24:07.514985 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.514960    2580 generic.go:356] "Generic (PLEG): container finished" podID="69a44edc-9973-43b2-94b2-2170164f0987" containerID="1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58" exitCode=0
Sep 10 21:24:07.515084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.514993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"
Sep 10 21:24:07.515084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.515039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" event={"ID":"69a44edc-9973-43b2-94b2-2170164f0987","Type":"ContainerDied","Data":"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"}
Sep 10 21:24:07.515084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.515081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k" event={"ID":"69a44edc-9973-43b2-94b2-2170164f0987","Type":"ContainerDied","Data":"f3c469058833fdebd06ed2e26ae05336f993f58743b555152fb0888ee9eb3cf5"}
Sep 10 21:24:07.523266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.523147    2580 scope.go:117] "RemoveContainer" containerID="ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"
Sep 10 21:24:07.523499 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:07.523481    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025\": container with ID starting with ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025 not found: ID does not exist" containerID="ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"
Sep 10 21:24:07.523589 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.523511    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025"} err="failed to get container status \"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025\": rpc error: code = NotFound desc = could not find container \"ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025\": container with ID starting with ca9dd48122dfe1295a44b55ccc9ff78d4cb37ffe98c0422c075d1a4494de6025 not found: ID does not exist"
Sep 10 21:24:07.523589 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.523534    2580 scope.go:117] "RemoveContainer" containerID="1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"
Sep 10 21:24:07.527627 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.527597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:24:07.531499 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.531483    2580 scope.go:117] "RemoveContainer" containerID="1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"
Sep 10 21:24:07.531735 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:07.531719    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58\": container with ID starting with 1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58 not found: ID does not exist" containerID="1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"
Sep 10 21:24:07.531786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.531743    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58"} err="failed to get container status \"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58\": rpc error: code = NotFound desc = could not find container \"1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58\": container with ID starting with 1069cb5ec465bb166f6f8a6245fe1ca59b292eac7156274d585468a026a79b58 not found: ID does not exist"
Sep 10 21:24:07.538900 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.538880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:24:07.543383 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.543352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-780a0cdd-77fd86c989-8825k"]
Sep 10 21:24:07.553601 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.553581    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:24:07.561439 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:07.561420    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-780a0cdd-6f7bb84bb4-682dj"]
Sep 10 21:24:08.612479 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:08.612447    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a44edc-9973-43b2-94b2-2170164f0987" path="/var/lib/kubelet/pods/69a44edc-9973-43b2-94b2-2170164f0987/volumes"
Sep 10 21:24:08.618693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:08.613362    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" path="/var/lib/kubelet/pods/b8bd763b-9899-4063-ab9f-4c256dad1c5b/volumes"
Sep 10 21:24:11.574159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574128    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:11.574706 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574687    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" containerName="maas-api"
Sep 10 21:24:11.574788 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574709    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" containerName="maas-api"
Sep 10 21:24:11.574788 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574744    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a44edc-9973-43b2-94b2-2170164f0987" containerName="payload-processing"
Sep 10 21:24:11.574788 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574755    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a44edc-9973-43b2-94b2-2170164f0987" containerName="payload-processing"
Sep 10 21:24:11.574941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574858    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8bd763b-9899-4063-ab9f-4c256dad1c5b" containerName="maas-api"
Sep 10 21:24:11.574941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.574874    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a44edc-9973-43b2-94b2-2170164f0987" containerName="payload-processing"
Sep 10 21:24:11.578137 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.578115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.580216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.580199    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-23d91506-serving-cert\""
Sep 10 21:24:11.587876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.587846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:11.647930 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.647901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.648067 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.647946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h25q\" (UniqueName: \"kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.648067 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.647975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.648067 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.648044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.748787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.748753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.748957 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.748819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.748957 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.748844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.748957 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.748880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h25q\" (UniqueName: \"kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.749118 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:11.748975    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-serving-cert: secret "maas-api-e2e-migrate-23d91506-serving-cert" not found
Sep 10 21:24:11.749118 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:11.749052    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls podName:b55dfebb-bbaf-41b4-a66c-b896c786c971 nodeName:}" failed. No retries permitted until 2026-09-10 21:24:12.249033737 +0000 UTC m=+870.127742782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls") pod "maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" (UID: "b55dfebb-bbaf-41b4-a66c-b896c786c971") : secret "maas-api-e2e-migrate-23d91506-serving-cert" not found
Sep 10 21:24:11.749557 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.749537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.751172 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.751152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:11.762461 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:11.762437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h25q\" (UniqueName: \"kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:12.253647 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.253608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:12.256152 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.256119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") pod \"maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:12.336517 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.336487    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:12.340147 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.340131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.342918 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.342894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-23d91506\""
Sep 10 21:24:12.343157 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.343139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-23d91506-dockercfg-8hhl5\""
Sep 10 21:24:12.351330 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.351308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:12.384011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.383982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:12.387342 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.387324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.400231 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.400212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:12.455537 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.455503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.455714 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.455568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x62x\" (UniqueName: \"kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.455789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.455712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr67p\" (UniqueName: \"kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.455789 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.455755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.489225 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.489194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.557766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x62x\" (UniqueName: \"kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.557872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr67p\" (UniqueName: \"kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.557910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.558000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.558839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.560175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.560113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.568455 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.568433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x62x\" (UniqueName: \"kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x\") pod \"payload-processing-e2e-migrate-23d91506-6797b5566-v58xb\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") " pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.568887 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.568777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr67p\" (UniqueName: \"kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p\") pod \"payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.652192 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.652167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:12.697649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.697623    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:12.781387 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.781305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:12.782203 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:12.782172    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55dfebb_bbaf_41b4_a66c_b896c786c971.slice/crio-a3353d8ec69e37ae49b2ad5a24ad9b7a10d77e6113e4c9e22b118c72b6f8c270 WatchSource:0}: Error finding container a3353d8ec69e37ae49b2ad5a24ad9b7a10d77e6113e4c9e22b118c72b6f8c270: Status 404 returned error can't find the container with id a3353d8ec69e37ae49b2ad5a24ad9b7a10d77e6113e4c9e22b118c72b6f8c270
Sep 10 21:24:12.931060 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.931037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:12.932235 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:12.932209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf95f2d6_4e42_480b_9178_c733ce7ddbcd.slice/crio-ff276ba98ca481f70b959c44e2a6af242fb0e0af0c96deb0145c86081b009efb WatchSource:0}: Error finding container ff276ba98ca481f70b959c44e2a6af242fb0e0af0c96deb0145c86081b009efb: Status 404 returned error can't find the container with id ff276ba98ca481f70b959c44e2a6af242fb0e0af0c96deb0145c86081b009efb
Sep 10 21:24:12.978476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:12.978452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:12.981195 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:12.981174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e20ff2_f903_45e3_b8a0_ff596d8cd0ee.slice/crio-4d20ecac41413a2324d14dd655d47d309ccdca8bfb5880267c51b87b609e43d5 WatchSource:0}: Error finding container 4d20ecac41413a2324d14dd655d47d309ccdca8bfb5880267c51b87b609e43d5: Status 404 returned error can't find the container with id 4d20ecac41413a2324d14dd655d47d309ccdca8bfb5880267c51b87b609e43d5
Sep 10 21:24:13.552180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.552085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" event={"ID":"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee","Type":"ContainerStarted","Data":"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"}
Sep 10 21:24:13.552180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.552177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" event={"ID":"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee","Type":"ContainerStarted","Data":"4d20ecac41413a2324d14dd655d47d309ccdca8bfb5880267c51b87b609e43d5"}
Sep 10 21:24:13.552435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.552390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:13.554423 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.554309    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" event={"ID":"cf95f2d6-4e42-480b-9178-c733ce7ddbcd","Type":"ContainerStarted","Data":"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"}
Sep 10 21:24:13.554423 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.554370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:13.554423 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.554395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" event={"ID":"cf95f2d6-4e42-480b-9178-c733ce7ddbcd","Type":"ContainerStarted","Data":"ff276ba98ca481f70b959c44e2a6af242fb0e0af0c96deb0145c86081b009efb"}
Sep 10 21:24:13.556477 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.556447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" event={"ID":"b55dfebb-bbaf-41b4-a66c-b896c786c971","Type":"ContainerStarted","Data":"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"}
Sep 10 21:24:13.556572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.556488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" event={"ID":"b55dfebb-bbaf-41b4-a66c-b896c786c971","Type":"ContainerStarted","Data":"a3353d8ec69e37ae49b2ad5a24ad9b7a10d77e6113e4c9e22b118c72b6f8c270"}
Sep 10 21:24:13.556572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.556514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:13.568821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.568776    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" podStartSLOduration=1.568759781 podStartE2EDuration="1.568759781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:13.567746787 +0000 UTC m=+871.446455850" watchObservedRunningTime="2026-09-10 21:24:13.568759781 +0000 UTC m=+871.447468847"
Sep 10 21:24:13.585333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.585292    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" podStartSLOduration=2.195586206 podStartE2EDuration="2.585261456s" totalImagesPullingTime="389.67525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:13.584218827 +0000 UTC m=+871.462927889" watchObservedRunningTime="2026-09-10 21:24:13.585261456 +0000 UTC m=+871.463970521"
Sep 10 21:24:13.603100 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:13.603053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" podStartSLOduration=1.603042394 podStartE2EDuration="1.603042394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:13.600147436 +0000 UTC m=+871.478856501" watchObservedRunningTime="2026-09-10 21:24:13.603042394 +0000 UTC m=+871.481751455"
Sep 10 21:24:19.567410 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:19.567382    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:24.564579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:24.564547    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:24.565019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:24.564632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:42.278008 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.277912    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:42.278587 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.278166    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" podUID="b55dfebb-bbaf-41b4-a66c-b896c786c971" containerName="maas-api" containerID="cri-o://05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63" gracePeriod=30
Sep 10 21:24:42.376729 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.376699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:42.376994 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.376946    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" podUID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" containerName="payload-processing" containerID="cri-o://3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6" gracePeriod=30
Sep 10 21:24:42.386169 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.386080    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:42.386374 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.386317    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" podUID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" containerName="payload-pre-processing" containerID="cri-o://6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853" gracePeriod=30
Sep 10 21:24:42.549431 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.549400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:42.610659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.610631    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:24:42.613043 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.613024    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:24:42.644019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.643996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs\") pod \"b55dfebb-bbaf-41b4-a66c-b896c786c971\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") "
Sep 10 21:24:42.644180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.644030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h25q\" (UniqueName: \"kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q\") pod \"b55dfebb-bbaf-41b4-a66c-b896c786c971\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") "
Sep 10 21:24:42.644180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.644082    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") pod \"b55dfebb-bbaf-41b4-a66c-b896c786c971\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") "
Sep 10 21:24:42.644180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.644132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca\") pod \"b55dfebb-bbaf-41b4-a66c-b896c786c971\" (UID: \"b55dfebb-bbaf-41b4-a66c-b896c786c971\") "
Sep 10 21:24:42.644563 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.644534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b55dfebb-bbaf-41b4-a66c-b896c786c971" (UID: "b55dfebb-bbaf-41b4-a66c-b896c786c971"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:42.646642 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.646614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q" (OuterVolumeSpecName: "kube-api-access-4h25q") pod "b55dfebb-bbaf-41b4-a66c-b896c786c971" (UID: "b55dfebb-bbaf-41b4-a66c-b896c786c971"). InnerVolumeSpecName "kube-api-access-4h25q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:42.647985 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.647959    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b55dfebb-bbaf-41b4-a66c-b896c786c971" (UID: "b55dfebb-bbaf-41b4-a66c-b896c786c971"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:42.649229 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.649204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b55dfebb-bbaf-41b4-a66c-b896c786c971" (UID: "b55dfebb-bbaf-41b4-a66c-b896c786c971"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:42.666194 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.666173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:42.674787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.674766    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:42.731150 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.731112    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" containerID="6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853" exitCode=0
Sep 10 21:24:42.731353 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.731161    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"
Sep 10 21:24:42.731353 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.731189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" event={"ID":"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee","Type":"ContainerDied","Data":"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"}
Sep 10 21:24:42.731353 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.731257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq" event={"ID":"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee","Type":"ContainerDied","Data":"4d20ecac41413a2324d14dd655d47d309ccdca8bfb5880267c51b87b609e43d5"}
Sep 10 21:24:42.731353 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.731313    2580 scope.go:117] "RemoveContainer" containerID="6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"
Sep 10 21:24:42.732881 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.732857    2580 generic.go:356] "Generic (PLEG): container finished" podID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" containerID="3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6" exitCode=0
Sep 10 21:24:42.732992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.732902    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"
Sep 10 21:24:42.732992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.732941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" event={"ID":"cf95f2d6-4e42-480b-9178-c733ce7ddbcd","Type":"ContainerDied","Data":"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"}
Sep 10 21:24:42.732992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.732978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb" event={"ID":"cf95f2d6-4e42-480b-9178-c733ce7ddbcd","Type":"ContainerDied","Data":"ff276ba98ca481f70b959c44e2a6af242fb0e0af0c96deb0145c86081b009efb"}
Sep 10 21:24:42.734798 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.734765    2580 generic.go:356] "Generic (PLEG): container finished" podID="b55dfebb-bbaf-41b4-a66c-b896c786c971" containerID="05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63" exitCode=0
Sep 10 21:24:42.734917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.734808    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"
Sep 10 21:24:42.734917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.734901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" event={"ID":"b55dfebb-bbaf-41b4-a66c-b896c786c971","Type":"ContainerDied","Data":"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"}
Sep 10 21:24:42.735031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.734942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75" event={"ID":"b55dfebb-bbaf-41b4-a66c-b896c786c971","Type":"ContainerDied","Data":"a3353d8ec69e37ae49b2ad5a24ad9b7a10d77e6113e4c9e22b118c72b6f8c270"}
Sep 10 21:24:42.741729 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.741711    2580 scope.go:117] "RemoveContainer" containerID="6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"
Sep 10 21:24:42.742027 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:42.742006    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853\": container with ID starting with 6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853 not found: ID does not exist" containerID="6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"
Sep 10 21:24:42.742108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.742034    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853"} err="failed to get container status \"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853\": rpc error: code = NotFound desc = could not find container \"6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853\": container with ID starting with 6f41597c518392fdb0f7fc6bdbabb6c0f73d6e0b13d44ec3978c67dfba256853 not found: ID does not exist"
Sep 10 21:24:42.742108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.742049    2580 scope.go:117] "RemoveContainer" containerID="3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"
Sep 10 21:24:42.744928 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.744911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x62x\" (UniqueName: \"kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x\") pod \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") "
Sep 10 21:24:42.745012 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.744936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr67p\" (UniqueName: \"kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p\") pod \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") "
Sep 10 21:24:42.745012 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.744970    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume\") pod \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\" (UID: \"cf95f2d6-4e42-480b-9178-c733ce7ddbcd\") "
Sep 10 21:24:42.745110 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume\") pod \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\" (UID: \"d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee\") "
Sep 10 21:24:42.745330 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745313    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.745412 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745340    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h25q\" (UniqueName: \"kubernetes.io/projected/b55dfebb-bbaf-41b4-a66c-b896c786c971-kube-api-access-4h25q\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.745412 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745358    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55dfebb-bbaf-41b4-a66c-b896c786c971-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.745412 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745373    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55dfebb-bbaf-41b4-a66c-b896c786c971-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.745571 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745419    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf95f2d6-4e42-480b-9178-c733ce7ddbcd" (UID: "cf95f2d6-4e42-480b-9178-c733ce7ddbcd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:42.745571 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.745441    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" (UID: "d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:42.747129 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.747103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p" (OuterVolumeSpecName: "kube-api-access-fr67p") pod "d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" (UID: "d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee"). InnerVolumeSpecName "kube-api-access-fr67p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:42.747259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.747239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x" (OuterVolumeSpecName: "kube-api-access-7x62x") pod "cf95f2d6-4e42-480b-9178-c733ce7ddbcd" (UID: "cf95f2d6-4e42-480b-9178-c733ce7ddbcd"). InnerVolumeSpecName "kube-api-access-7x62x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:42.750414 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.750397    2580 scope.go:117] "RemoveContainer" containerID="3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"
Sep 10 21:24:42.750652 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:42.750633    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6\": container with ID starting with 3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6 not found: ID does not exist" containerID="3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"
Sep 10 21:24:42.750727 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.750659    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6"} err="failed to get container status \"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6\": rpc error: code = NotFound desc = could not find container \"3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6\": container with ID starting with 3a423c2d07fca7dacc52021b75f83427b0ce588427822b274bd16bc4a5f8e2b6 not found: ID does not exist"
Sep 10 21:24:42.750727 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.750680    2580 scope.go:117] "RemoveContainer" containerID="05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"
Sep 10 21:24:42.756977 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.756955    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:42.759645 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.759627    2580 scope.go:117] "RemoveContainer" containerID="05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"
Sep 10 21:24:42.759918 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:42.759899    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63\": container with ID starting with 05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63 not found: ID does not exist" containerID="05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"
Sep 10 21:24:42.759974 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.759923    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63"} err="failed to get container status \"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63\": rpc error: code = NotFound desc = could not find container \"05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63\": container with ID starting with 05e1ed7aad16b76625797951d5ebcd58e2f32e5aae8ceb9af83e9e3ec56dbf63 not found: ID does not exist"
Sep 10 21:24:42.763359 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.763340    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-23d91506-6f57dd6dc6-rph75"]
Sep 10 21:24:42.846708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.846654    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.846708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.846675    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x62x\" (UniqueName: \"kubernetes.io/projected/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-kube-api-access-7x62x\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.846708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.846686    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr67p\" (UniqueName: \"kubernetes.io/projected/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee-kube-api-access-fr67p\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:42.846708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:42.846694    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf95f2d6-4e42-480b-9178-c733ce7ddbcd-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:24:43.052210 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:43.052178    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:43.055957 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:43.055916    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-23d91506-554f595ff4-pbxrq"]
Sep 10 21:24:43.066014 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:43.065971    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:43.071375 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:43.069629    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-23d91506-6797b5566-v58xb"]
Sep 10 21:24:44.611600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:44.611571    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55dfebb-bbaf-41b4-a66c-b896c786c971" path="/var/lib/kubelet/pods/b55dfebb-bbaf-41b4-a66c-b896c786c971/volumes"
Sep 10 21:24:44.611950 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:44.611908    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" path="/var/lib/kubelet/pods/cf95f2d6-4e42-480b-9178-c733ce7ddbcd/volumes"
Sep 10 21:24:44.612206 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:44.612193    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" path="/var/lib/kubelet/pods/d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee/volumes"
Sep 10 21:24:45.189618 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:45.189581    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:24:47.593545 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593514    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593907    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" containerName="payload-pre-processing"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593919    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" containerName="payload-pre-processing"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593929    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55dfebb-bbaf-41b4-a66c-b896c786c971" containerName="maas-api"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593935    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55dfebb-bbaf-41b4-a66c-b896c786c971" containerName="maas-api"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593956    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" containerName="payload-processing"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.593963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" containerName="payload-processing"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.594014    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e20ff2-f903-45e3-b8a0-ff596d8cd0ee" containerName="payload-pre-processing"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.594022    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55dfebb-bbaf-41b4-a66c-b896c786c971" containerName="maas-api"
Sep 10 21:24:47.594023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.594029    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf95f2d6-4e42-480b-9178-c733ce7ddbcd" containerName="payload-processing"
Sep 10 21:24:47.598774 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.598755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.600989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.600969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dc873af1-serving-cert\""
Sep 10 21:24:47.609255 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.609231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:24:47.691096 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.691066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.691267 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.691104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fn9\" (UniqueName: \"kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.691267 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.691190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.691267 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.691246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.792699 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.792668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.792845 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.792712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fn9\" (UniqueName: \"kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.792845 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.792784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.792960 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.792843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.792960 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:47.792948    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-serving-cert: secret "maas-api-e2e-ait-dc873af1-serving-cert" not found
Sep 10 21:24:47.793059 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:47.793017    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls podName:09fc4d58-c487-40e1-aea9-b4c10eece920 nodeName:}" failed. No retries permitted until 2026-09-10 21:24:48.293000926 +0000 UTC m=+906.171709966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls") pod "maas-api-e2e-ait-dc873af1-57864db544-tj2xq" (UID: "09fc4d58-c487-40e1-aea9-b4c10eece920") : secret "maas-api-e2e-ait-dc873af1-serving-cert" not found
Sep 10 21:24:47.793483 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.793462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.795099 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.795081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:47.804307 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:47.804286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fn9\" (UniqueName: \"kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:48.297010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.296972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:48.299230 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.299202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") pod \"maas-api-e2e-ait-dc873af1-57864db544-tj2xq\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:48.334339 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.334308    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:24:48.338223 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.338203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.341418 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.341399    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-dc873af1\""
Sep 10 21:24:48.351024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.350999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:24:48.366534 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.366508    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:24:48.370022 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.370002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.372587 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.372565    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-dc873af1-dockercfg-8w42l\""
Sep 10 21:24:48.379166 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.379146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:24:48.397981 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.397956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.398073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.398007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjx8s\" (UniqueName: \"kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.435253 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:48.435230    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:24:48.498982 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.498954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.499083 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.498986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j9bb\" (UniqueName: \"kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.499129 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.499087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.499129 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.499122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjx8s\" (UniqueName: \"kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.499766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.499749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.507119 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.507098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjx8s\" (UniqueName: \"kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s\") pod \"payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.509872 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.509858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:48.600257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.600228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.600661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.600302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j9bb\" (UniqueName: \"kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.600886 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.600868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.608025 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.608000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j9bb\" (UniqueName: \"kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb\") pod \"payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") " pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.649158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.649137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:48.681745 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.681719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:48.784007 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.783980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:24:48.785705 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:48.785682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fc4d58_c487_40e1_aea9_b4c10eece920.slice/crio-e6f5a5e7c42908e8a20e89540dac20cb9429311087327fa6526b8c6d207840b9 WatchSource:0}: Error finding container e6f5a5e7c42908e8a20e89540dac20cb9429311087327fa6526b8c6d207840b9: Status 404 returned error can't find the container with id e6f5a5e7c42908e8a20e89540dac20cb9429311087327fa6526b8c6d207840b9
Sep 10 21:24:48.927709 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.927682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:24:48.929037 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:48.929010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049d9233_873b_4c2c_8a4a_14acea93e43d.slice/crio-bec04be3a3cfe08eb8ce25cf90b468aa472084b7119869aa157ab30542d0b578 WatchSource:0}: Error finding container bec04be3a3cfe08eb8ce25cf90b468aa472084b7119869aa157ab30542d0b578: Status 404 returned error can't find the container with id bec04be3a3cfe08eb8ce25cf90b468aa472084b7119869aa157ab30542d0b578
Sep 10 21:24:48.962879 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:48.962806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:24:48.965210 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:24:48.965183    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8d20b8_80b2_4932_b84e_302c1e304354.slice/crio-74badab1aa2c82079653b978d5ada77d6d649f38f0e786293746c651faf782ba WatchSource:0}: Error finding container 74badab1aa2c82079653b978d5ada77d6d649f38f0e786293746c651faf782ba: Status 404 returned error can't find the container with id 74badab1aa2c82079653b978d5ada77d6d649f38f0e786293746c651faf782ba
Sep 10 21:24:49.779084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.779045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" event={"ID":"09fc4d58-c487-40e1-aea9-b4c10eece920","Type":"ContainerStarted","Data":"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"}
Sep 10 21:24:49.779583 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.779123    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:24:49.779583 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.779152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" event={"ID":"09fc4d58-c487-40e1-aea9-b4c10eece920","Type":"ContainerStarted","Data":"e6f5a5e7c42908e8a20e89540dac20cb9429311087327fa6526b8c6d207840b9"}
Sep 10 21:24:49.780719 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.780689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" event={"ID":"ad8d20b8-80b2-4932-b84e-302c1e304354","Type":"ContainerStarted","Data":"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"}
Sep 10 21:24:49.780847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.780745    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:24:49.780847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.780770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" event={"ID":"ad8d20b8-80b2-4932-b84e-302c1e304354","Type":"ContainerStarted","Data":"74badab1aa2c82079653b978d5ada77d6d649f38f0e786293746c651faf782ba"}
Sep 10 21:24:49.782301 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.782251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" event={"ID":"049d9233-873b-4c2c-8a4a-14acea93e43d","Type":"ContainerStarted","Data":"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"}
Sep 10 21:24:49.782439 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.782323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" event={"ID":"049d9233-873b-4c2c-8a4a-14acea93e43d","Type":"ContainerStarted","Data":"bec04be3a3cfe08eb8ce25cf90b468aa472084b7119869aa157ab30542d0b578"}
Sep 10 21:24:49.782439 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.782417    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:24:49.796924 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.796882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" podStartSLOduration=2.354118265 podStartE2EDuration="2.796866183s" totalImagesPullingTime="442.747918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:49.796208687 +0000 UTC m=+907.674917749" watchObservedRunningTime="2026-09-10 21:24:49.796866183 +0000 UTC m=+907.675575247"
Sep 10 21:24:49.813509 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.813467    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" podStartSLOduration=1.813456606 podStartE2EDuration="1.813456606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:49.811226401 +0000 UTC m=+907.689935466" watchObservedRunningTime="2026-09-10 21:24:49.813456606 +0000 UTC m=+907.692165669"
Sep 10 21:24:49.826746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:49.826710    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" podStartSLOduration=1.826697284 podStartE2EDuration="1.826697284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:49.825168175 +0000 UTC m=+907.703877248" watchObservedRunningTime="2026-09-10 21:24:49.826697284 +0000 UTC m=+907.705406345"
Sep 10 21:24:55.238816 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:24:55.238784    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:24:55.794748 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:24:55.794720    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:25:00.790251 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:00.790210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:25:00.790736 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:00.790304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:25:03.379055 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.379023    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:25:03.379466 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.379237    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" podUID="c7bc986c-b502-48a9-ba30-4947a0108fc2" containerName="maas-api" containerID="cri-o://bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672" gracePeriod=30
Sep 10 21:25:03.494920 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:03.494891    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:03.631456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.631400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:25:03.743652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.743623    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca\") pod \"c7bc986c-b502-48a9-ba30-4947a0108fc2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") "
Sep 10 21:25:03.743836 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.743682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs\") pod \"c7bc986c-b502-48a9-ba30-4947a0108fc2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") "
Sep 10 21:25:03.743836 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.743715    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") pod \"c7bc986c-b502-48a9-ba30-4947a0108fc2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") "
Sep 10 21:25:03.743836 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.743797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7pph\" (UniqueName: \"kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph\") pod \"c7bc986c-b502-48a9-ba30-4947a0108fc2\" (UID: \"c7bc986c-b502-48a9-ba30-4947a0108fc2\") "
Sep 10 21:25:03.744026 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.744007    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7bc986c-b502-48a9-ba30-4947a0108fc2" (UID: "c7bc986c-b502-48a9-ba30-4947a0108fc2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:03.744735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.744557    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7bc986c-b502-48a9-ba30-4947a0108fc2-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.746017 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.745987    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph" (OuterVolumeSpecName: "kube-api-access-l7pph") pod "c7bc986c-b502-48a9-ba30-4947a0108fc2" (UID: "c7bc986c-b502-48a9-ba30-4947a0108fc2"). InnerVolumeSpecName "kube-api-access-l7pph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:03.746288 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.746245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c7bc986c-b502-48a9-ba30-4947a0108fc2" (UID: "c7bc986c-b502-48a9-ba30-4947a0108fc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:03.746370 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.746305    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7bc986c-b502-48a9-ba30-4947a0108fc2" (UID: "c7bc986c-b502-48a9-ba30-4947a0108fc2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:03.845653 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.845626    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.845653 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.845650    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7bc986c-b502-48a9-ba30-4947a0108fc2-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.845824 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.845659    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7pph\" (UniqueName: \"kubernetes.io/projected/c7bc986c-b502-48a9-ba30-4947a0108fc2-kube-api-access-l7pph\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.858614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.858580    2580 generic.go:356] "Generic (PLEG): container finished" podID="c7bc986c-b502-48a9-ba30-4947a0108fc2" containerID="bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672" exitCode=0
Sep 10 21:25:03.858769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.858623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" event={"ID":"c7bc986c-b502-48a9-ba30-4947a0108fc2","Type":"ContainerDied","Data":"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"}
Sep 10 21:25:03.858769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.858633    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"
Sep 10 21:25:03.858769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.858670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2" event={"ID":"c7bc986c-b502-48a9-ba30-4947a0108fc2","Type":"ContainerDied","Data":"e4b0341967c680a76d6967415ea2a3616c29ea26e14c6fb8d0c891e2d30dcd34"}
Sep 10 21:25:03.858769 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.858692    2580 scope.go:117] "RemoveContainer" containerID="bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"
Sep 10 21:25:03.867833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.867801    2580 scope.go:117] "RemoveContainer" containerID="bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"
Sep 10 21:25:03.868162 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:03.868138    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672\": container with ID starting with bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672 not found: ID does not exist" containerID="bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"
Sep 10 21:25:03.868252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.868173    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672"} err="failed to get container status \"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672\": rpc error: code = NotFound desc = could not find container \"bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672\": container with ID starting with bfaf4e824bf89e7e6e366678738e6d56a41999d663c90989ea609385baa29672 not found: ID does not exist"
Sep 10 21:25:03.880328 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.880307    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:25:03.884290 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:03.884218    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f135f4-766b7fbc87-qdzg2"]
Sep 10 21:25:04.611684 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:04.611648    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7bc986c-b502-48a9-ba30-4947a0108fc2" path="/var/lib/kubelet/pods/c7bc986c-b502-48a9-ba30-4947a0108fc2/volumes"
Sep 10 21:25:05.248280 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:05.248232    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:06.499312 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:06.499254    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:06.500097 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:06.500066    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:12.020717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.020685    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"]
Sep 10 21:25:12.021291 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.021254    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7bc986c-b502-48a9-ba30-4947a0108fc2" containerName="maas-api"
Sep 10 21:25:12.021291 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.021291    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7bc986c-b502-48a9-ba30-4947a0108fc2" containerName="maas-api"
Sep 10 21:25:12.021454 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.021440    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7bc986c-b502-48a9-ba30-4947a0108fc2" containerName="maas-api"
Sep 10 21:25:12.024473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.024452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:12.026481 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.026461    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:25:12.031142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.031122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"]
Sep 10 21:25:12.120808 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.120781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl\" (UID: \"7a7c08f3-04a0-4aca-a22e-8037d05556a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:12.221194 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.221164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl\" (UID: \"7a7c08f3-04a0-4aca-a22e-8037d05556a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:12.221780 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.221761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl\" (UID: \"7a7c08f3-04a0-4aca-a22e-8037d05556a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:12.336175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.336116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:12.604484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.604458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"]
Sep 10 21:25:12.607020 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:12.606988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7c08f3_04a0_4aca_a22e_8037d05556a9.slice/crio-1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134 WatchSource:0}: Error finding container 1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134: Status 404 returned error can't find the container with id 1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134
Sep 10 21:25:12.907000 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:12.906964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl" event={"ID":"7a7c08f3-04a0-4aca-a22e-8037d05556a9","Type":"ContainerStarted","Data":"1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134"}
Sep 10 21:25:14.920085 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:14.920056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl" event={"ID":"7a7c08f3-04a0-4aca-a22e-8037d05556a9","Type":"ContainerStarted","Data":"5a7d034bb89ee9531cdfa64fbd5c518503919842fd03056bdf6d3c2642e43448"}
Sep 10 21:25:14.935297 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:14.935235    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl" podStartSLOduration=0.85500103 podStartE2EDuration="2.935218503s" totalImagesPullingTime="2.080217473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:14.934009341 +0000 UTC m=+932.812718404" watchObservedRunningTime="2026-09-10 21:25:14.935218503 +0000 UTC m=+932.813927564"
Sep 10 21:25:15.299216 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:15.299187    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:15.926553 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:15.926524    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a7c08f3-04a0-4aca-a22e-8037d05556a9" containerID="5a7d034bb89ee9531cdfa64fbd5c518503919842fd03056bdf6d3c2642e43448" exitCode=0
Sep 10 21:25:15.926973 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:15.926609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl" event={"ID":"7a7c08f3-04a0-4aca-a22e-8037d05556a9","Type":"ContainerDied","Data":"5a7d034bb89ee9531cdfa64fbd5c518503919842fd03056bdf6d3c2642e43448"}
Sep 10 21:25:17.067315 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.067293    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:17.162649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.162618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca\") pod \"7a7c08f3-04a0-4aca-a22e-8037d05556a9\" (UID: \"7a7c08f3-04a0-4aca-a22e-8037d05556a9\") "
Sep 10 21:25:17.162958 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.162937    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a7c08f3-04a0-4aca-a22e-8037d05556a9" (UID: "7a7c08f3-04a0-4aca-a22e-8037d05556a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:17.263707 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.263640    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a7c08f3-04a0-4aca-a22e-8037d05556a9-maas-api-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:17.939011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.938986    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"
Sep 10 21:25:17.939178 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.939024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl" event={"ID":"7a7c08f3-04a0-4aca-a22e-8037d05556a9","Type":"ContainerDied","Data":"1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134"}
Sep 10 21:25:17.939178 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:17.939074    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e98452c6434df49a6661cee5cdc08de1008d7483234d5ddccfb1c3d0ecf7134"
Sep 10 21:25:18.445098 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:18.445049    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:25.347143 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:25.347109    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:28.412927 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.412897    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:25:28.413361 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.413099    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" podUID="09fc4d58-c487-40e1-aea9-b4c10eece920" containerName="maas-api" containerID="cri-o://ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505" gracePeriod=30
Sep 10 21:25:28.471893 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.471862    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:25:28.472074 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.472049    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" podUID="ad8d20b8-80b2-4932-b84e-302c1e304354" containerName="payload-processing" containerID="cri-o://a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c" gracePeriod=30
Sep 10 21:25:28.479738 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.479592    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:25:28.480090 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.479831    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" podUID="049d9233-873b-4c2c-8a4a-14acea93e43d" containerName="payload-pre-processing" containerID="cri-o://9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe" gracePeriod=30
Sep 10 21:25:28.650114 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.650084    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"]
Sep 10 21:25:28.655730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.655606    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc873af1-pfwnl"]
Sep 10 21:25:28.745765 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.745738    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:25:28.768866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.768628    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:25:28.779648 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.779629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:25:28.872095 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j9bb\" (UniqueName: \"kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb\") pod \"ad8d20b8-80b2-4932-b84e-302c1e304354\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") "
Sep 10 21:25:28.872259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") pod \"09fc4d58-c487-40e1-aea9-b4c10eece920\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") "
Sep 10 21:25:28.872259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872153    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4fn9\" (UniqueName: \"kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9\") pod \"09fc4d58-c487-40e1-aea9-b4c10eece920\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") "
Sep 10 21:25:28.872259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume\") pod \"ad8d20b8-80b2-4932-b84e-302c1e304354\" (UID: \"ad8d20b8-80b2-4932-b84e-302c1e304354\") "
Sep 10 21:25:28.872259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872212    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume\") pod \"049d9233-873b-4c2c-8a4a-14acea93e43d\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") "
Sep 10 21:25:28.872259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjx8s\" (UniqueName: \"kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s\") pod \"049d9233-873b-4c2c-8a4a-14acea93e43d\" (UID: \"049d9233-873b-4c2c-8a4a-14acea93e43d\") "
Sep 10 21:25:28.872566 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872469    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca\") pod \"09fc4d58-c487-40e1-aea9-b4c10eece920\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") "
Sep 10 21:25:28.872566 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872545    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs\") pod \"09fc4d58-c487-40e1-aea9-b4c10eece920\" (UID: \"09fc4d58-c487-40e1-aea9-b4c10eece920\") "
Sep 10 21:25:28.872674 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872559    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "049d9233-873b-4c2c-8a4a-14acea93e43d" (UID: "049d9233-873b-4c2c-8a4a-14acea93e43d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:28.872674 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872631    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad8d20b8-80b2-4932-b84e-302c1e304354" (UID: "ad8d20b8-80b2-4932-b84e-302c1e304354"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:28.872920 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "09fc4d58-c487-40e1-aea9-b4c10eece920" (UID: "09fc4d58-c487-40e1-aea9-b4c10eece920"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:28.872920 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872911    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8d20b8-80b2-4932-b84e-302c1e304354-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.873039 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.872933    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/049d9233-873b-4c2c-8a4a-14acea93e43d-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.874098 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.874074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb" (OuterVolumeSpecName: "kube-api-access-2j9bb") pod "ad8d20b8-80b2-4932-b84e-302c1e304354" (UID: "ad8d20b8-80b2-4932-b84e-302c1e304354"). InnerVolumeSpecName "kube-api-access-2j9bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:28.874283 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.874258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9" (OuterVolumeSpecName: "kube-api-access-k4fn9") pod "09fc4d58-c487-40e1-aea9-b4c10eece920" (UID: "09fc4d58-c487-40e1-aea9-b4c10eece920"). InnerVolumeSpecName "kube-api-access-k4fn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:28.874617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.874601    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "09fc4d58-c487-40e1-aea9-b4c10eece920" (UID: "09fc4d58-c487-40e1-aea9-b4c10eece920"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:28.874673 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.874625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "09fc4d58-c487-40e1-aea9-b4c10eece920" (UID: "09fc4d58-c487-40e1-aea9-b4c10eece920"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:28.874757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.874741    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s" (OuterVolumeSpecName: "kube-api-access-qjx8s") pod "049d9233-873b-4c2c-8a4a-14acea93e43d" (UID: "049d9233-873b-4c2c-8a4a-14acea93e43d"). InnerVolumeSpecName "kube-api-access-qjx8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974384    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974408    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4fn9\" (UniqueName: \"kubernetes.io/projected/09fc4d58-c487-40e1-aea9-b4c10eece920-kube-api-access-k4fn9\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974418    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjx8s\" (UniqueName: \"kubernetes.io/projected/049d9233-873b-4c2c-8a4a-14acea93e43d-kube-api-access-qjx8s\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974428    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fc4d58-c487-40e1-aea9-b4c10eece920-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974436    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09fc4d58-c487-40e1-aea9-b4c10eece920-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.974451 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.974446    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j9bb\" (UniqueName: \"kubernetes.io/projected/ad8d20b8-80b2-4932-b84e-302c1e304354-kube-api-access-2j9bb\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:25:28.998060 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.998030    2580 generic.go:356] "Generic (PLEG): container finished" podID="09fc4d58-c487-40e1-aea9-b4c10eece920" containerID="ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505" exitCode=0
Sep 10 21:25:28.998193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.998082    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"
Sep 10 21:25:28.998193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.998125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" event={"ID":"09fc4d58-c487-40e1-aea9-b4c10eece920","Type":"ContainerDied","Data":"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"}
Sep 10 21:25:28.998193 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.998174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq" event={"ID":"09fc4d58-c487-40e1-aea9-b4c10eece920","Type":"ContainerDied","Data":"e6f5a5e7c42908e8a20e89540dac20cb9429311087327fa6526b8c6d207840b9"}
Sep 10 21:25:28.998394 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.998195    2580 scope.go:117] "RemoveContainer" containerID="ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"
Sep 10 21:25:28.999730 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.999705    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad8d20b8-80b2-4932-b84e-302c1e304354" containerID="a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c" exitCode=0
Sep 10 21:25:28.999827 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.999736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" event={"ID":"ad8d20b8-80b2-4932-b84e-302c1e304354","Type":"ContainerDied","Data":"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"}
Sep 10 21:25:28.999827 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.999801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg" event={"ID":"ad8d20b8-80b2-4932-b84e-302c1e304354","Type":"ContainerDied","Data":"74badab1aa2c82079653b978d5ada77d6d649f38f0e786293746c651faf782ba"}
Sep 10 21:25:28.999827 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:28.999756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"
Sep 10 21:25:29.001627 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.001581    2580 generic.go:356] "Generic (PLEG): container finished" podID="049d9233-873b-4c2c-8a4a-14acea93e43d" containerID="9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe" exitCode=0
Sep 10 21:25:29.001627 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.001614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" event={"ID":"049d9233-873b-4c2c-8a4a-14acea93e43d","Type":"ContainerDied","Data":"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"}
Sep 10 21:25:29.001761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.001632    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"
Sep 10 21:25:29.001761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.001656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t" event={"ID":"049d9233-873b-4c2c-8a4a-14acea93e43d","Type":"ContainerDied","Data":"bec04be3a3cfe08eb8ce25cf90b468aa472084b7119869aa157ab30542d0b578"}
Sep 10 21:25:29.009607 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.009587    2580 scope.go:117] "RemoveContainer" containerID="ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"
Sep 10 21:25:29.009958 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:29.009923    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505\": container with ID starting with ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505 not found: ID does not exist" containerID="ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"
Sep 10 21:25:29.010052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.009971    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505"} err="failed to get container status \"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505\": rpc error: code = NotFound desc = could not find container \"ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505\": container with ID starting with ce93decd9f62c308ab93e357d2c866177f26b6ba22df85612d440000b542f505 not found: ID does not exist"
Sep 10 21:25:29.010052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.009994    2580 scope.go:117] "RemoveContainer" containerID="a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"
Sep 10 21:25:29.017899 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.017885    2580 scope.go:117] "RemoveContainer" containerID="a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"
Sep 10 21:25:29.018171 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:29.018155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c\": container with ID starting with a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c not found: ID does not exist" containerID="a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"
Sep 10 21:25:29.018215 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.018179    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c"} err="failed to get container status \"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c\": rpc error: code = NotFound desc = could not find container \"a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c\": container with ID starting with a8924ad523508b2a11eb1ce79f9736c8a787747005fd7ced84f0135d8fc6947c not found: ID does not exist"
Sep 10 21:25:29.018215 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.018194    2580 scope.go:117] "RemoveContainer" containerID="9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"
Sep 10 21:25:29.022178 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.022159    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:25:29.028484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.028462    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc873af1-57864db544-tj2xq"]
Sep 10 21:25:29.030631 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.030609    2580 scope.go:117] "RemoveContainer" containerID="9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"
Sep 10 21:25:29.031029 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:29.030992    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe\": container with ID starting with 9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe not found: ID does not exist" containerID="9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"
Sep 10 21:25:29.031123 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.031039    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe"} err="failed to get container status \"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe\": rpc error: code = NotFound desc = could not find container \"9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe\": container with ID starting with 9de96c30c7a7879e50be23fb14cdfb435cb5960973ba9982695d99d01595e8fe not found: ID does not exist"
Sep 10 21:25:29.040890 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.040870    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:25:29.047249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.047231    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc873af1-5cd8d6f4df-drwqg"]
Sep 10 21:25:29.058333 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.058312    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:25:29.062950 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:29.062931    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc873af1-69988ccc6b-kkf4t"]
Sep 10 21:25:30.611490 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:30.611458    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049d9233-873b-4c2c-8a4a-14acea93e43d" path="/var/lib/kubelet/pods/049d9233-873b-4c2c-8a4a-14acea93e43d/volumes"
Sep 10 21:25:30.611870 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:30.611846    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09fc4d58-c487-40e1-aea9-b4c10eece920" path="/var/lib/kubelet/pods/09fc4d58-c487-40e1-aea9-b4c10eece920/volumes"
Sep 10 21:25:30.612120 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:30.612108    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a7c08f3-04a0-4aca-a22e-8037d05556a9" path="/var/lib/kubelet/pods/7a7c08f3-04a0-4aca-a22e-8037d05556a9/volumes"
Sep 10 21:25:30.612443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:30.612431    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad8d20b8-80b2-4932-b84e-302c1e304354" path="/var/lib/kubelet/pods/ad8d20b8-80b2-4932-b84e-302c1e304354/volumes"
Sep 10 21:25:33.303598 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303567    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:25:33.303953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303932    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="049d9233-873b-4c2c-8a4a-14acea93e43d" containerName="payload-pre-processing"
Sep 10 21:25:33.303953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303943    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="049d9233-873b-4c2c-8a4a-14acea93e43d" containerName="payload-pre-processing"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303960    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a7c08f3-04a0-4aca-a22e-8037d05556a9" containerName="revoke-keys"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303966    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a7c08f3-04a0-4aca-a22e-8037d05556a9" containerName="revoke-keys"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303978    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fc4d58-c487-40e1-aea9-b4c10eece920" containerName="maas-api"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303983    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fc4d58-c487-40e1-aea9-b4c10eece920" containerName="maas-api"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303992    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad8d20b8-80b2-4932-b84e-302c1e304354" containerName="payload-processing"
Sep 10 21:25:33.304024 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.303998    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad8d20b8-80b2-4932-b84e-302c1e304354" containerName="payload-processing"
Sep 10 21:25:33.304202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.304055    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a7c08f3-04a0-4aca-a22e-8037d05556a9" containerName="revoke-keys"
Sep 10 21:25:33.304202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.304062    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09fc4d58-c487-40e1-aea9-b4c10eece920" containerName="maas-api"
Sep 10 21:25:33.304202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.304073    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="049d9233-873b-4c2c-8a4a-14acea93e43d" containerName="payload-pre-processing"
Sep 10 21:25:33.304202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.304079    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad8d20b8-80b2-4932-b84e-302c1e304354" containerName="payload-processing"
Sep 10 21:25:33.308843 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.308824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.313395 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.313376    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5ec041\"/\"default-dockercfg-qh7kx\""
Sep 10 21:25:33.313521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.313442    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5ec041\"/\"openshift-service-ca.crt\""
Sep 10 21:25:33.313521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.313481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5ec041\"/\"kube-root-ca.crt\""
Sep 10 21:25:33.313521 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.313505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5ec041\"/\"test-model-5ec041-kserve-self-signed-certs\""
Sep 10 21:25:33.318593 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.318574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:25:33.412096 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.412225 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.412225 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.412322 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.412360 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.412396 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.412364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f5c4\" (UniqueName: \"kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.491206 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:33.491179    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:33.513010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.512983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513142 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f5c4\" (UniqueName: \"kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513522 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513604 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.513659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.513603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.515311 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.515291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.515561 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.515544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.521196 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.521176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f5c4\" (UniqueName: \"kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4\") pod \"test-model-5ec041-kserve-775fc5b5df-455xn\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") " pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.619171 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.619099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:33.834098 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.834064    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:25:33.838631 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.838606    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:33.841347 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.841281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-fd195260-serving-cert\""
Sep 10 21:25:33.841347 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.841323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:25:33.841557 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.841416    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:25:33.842049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.842023    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:25:33.842161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.842029    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:25:33.856216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.851677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:25:33.898679 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.898654    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:25:33.901715 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:33.901689    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bfce600_d66f_400e_8cd8_98c33f56dfeb.slice/crio-8d715a214105ba5f9e73902aa5e551858844fc346dd6d1d5866bc6287bda318e WatchSource:0}: Error finding container 8d715a214105ba5f9e73902aa5e551858844fc346dd6d1d5866bc6287bda318e: Status 404 returned error can't find the container with id 8d715a214105ba5f9e73902aa5e551858844fc346dd6d1d5866bc6287bda318e
Sep 10 21:25:33.916586 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.916558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:33.916754 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.916700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:33.916754 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.916741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtxgl\" (UniqueName: \"kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:33.916866 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:33.916772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.018069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.018036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.018248 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.018076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtxgl\" (UniqueName: \"kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.018248 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.018106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.018248 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.018162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.018408 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:34.018295    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-serving-cert: secret "maas-api-e2e-derive-fd195260-serving-cert" not found
Sep 10 21:25:34.018408 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:34.018373    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls podName:d33b2d4a-f1d4-4837-9c12-f277da7a4eda nodeName:}" failed. No retries permitted until 2026-09-10 21:25:34.518356985 +0000 UTC m=+952.397066025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls") pod "maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda") : secret "maas-api-e2e-derive-fd195260-serving-cert" not found
Sep 10 21:25:34.018968 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.018940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.020428 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.020410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.028779 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.028757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtxgl\" (UniqueName: \"kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.032754 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.032694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" event={"ID":"1bfce600-d66f-400e-8cd8-98c33f56dfeb","Type":"ContainerStarted","Data":"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"}
Sep 10 21:25:34.032754 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.032741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" event={"ID":"1bfce600-d66f-400e-8cd8-98c33f56dfeb","Type":"ContainerStarted","Data":"8d715a214105ba5f9e73902aa5e551858844fc346dd6d1d5866bc6287bda318e"}
Sep 10 21:25:34.053248 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.053199    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" podStartSLOduration=1.053184897 podStartE2EDuration="1.053184897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:34.051223614 +0000 UTC m=+951.929932676" watchObservedRunningTime="2026-09-10 21:25:34.053184897 +0000 UTC m=+951.931893964"
Sep 10 21:25:34.506555 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.506519    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:25:34.510415 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.510399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.512763 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.512744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-fd195260\""
Sep 10 21:25:34.518894 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.518870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:25:34.522022 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.522002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:34.522173 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:34.522155    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-serving-cert: secret "maas-api-e2e-derive-fd195260-serving-cert" not found
Sep 10 21:25:34.522237 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:34.522226    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls podName:d33b2d4a-f1d4-4837-9c12-f277da7a4eda nodeName:}" failed. No retries permitted until 2026-09-10 21:25:35.522205538 +0000 UTC m=+953.400914581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls") pod "maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda") : secret "maas-api-e2e-derive-fd195260-serving-cert" not found
Sep 10 21:25:34.530940 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.530917    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:25:34.534987 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.534970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.537410 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.537378    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-fd195260-dockercfg-vdbb8\""
Sep 10 21:25:34.543636 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.543613    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:25:34.623245 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.623214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.623430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.623260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.623430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.623353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkf69\" (UniqueName: \"kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.623430 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.623396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pntw\" (UniqueName: \"kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.724158 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkf69\" (UniqueName: \"kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.724319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pntw\" (UniqueName: \"kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.724319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.724319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.724906 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.724959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.724943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.732394 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.732376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pntw\" (UniqueName: \"kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw\") pod \"payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") " pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:34.732498 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.732420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkf69\" (UniqueName: \"kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69\") pod \"payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.834641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.834585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:34.846684 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:34.846641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:35.114391 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.114306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:25:35.116667 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:35.116641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8edd45a_dd7b_46f9_bb91_0dbbb87eecb2.slice/crio-89f4292d69b556cdcc5181b0fd5a6251fc47d4c4e538d8099312f8793bd275d0 WatchSource:0}: Error finding container 89f4292d69b556cdcc5181b0fd5a6251fc47d4c4e538d8099312f8793bd275d0: Status 404 returned error can't find the container with id 89f4292d69b556cdcc5181b0fd5a6251fc47d4c4e538d8099312f8793bd275d0
Sep 10 21:25:35.126749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.126705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:25:35.130857 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:35.130836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e5d478_c1df_443b_a257_77df79edb95a.slice/crio-1d010ba23821e412debedf74bc3b52891f9e496ad6fd3807ca8028f1fa3a6424 WatchSource:0}: Error finding container 1d010ba23821e412debedf74bc3b52891f9e496ad6fd3807ca8028f1fa3a6424: Status 404 returned error can't find the container with id 1d010ba23821e412debedf74bc3b52891f9e496ad6fd3807ca8028f1fa3a6424
Sep 10 21:25:35.359823 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:25:35.359787    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Sep 10 21:25:35.530522 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.530493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:35.533604 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.533582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:35.653246 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.653218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:35.928486 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:35.928460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:25:35.934620 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:35.934589    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33b2d4a_f1d4_4837_9c12_f277da7a4eda.slice/crio-0614fc056e0e8a06382460ea3293a46e9b93b76a243b92f04531fd6bf4ec06fa WatchSource:0}: Error finding container 0614fc056e0e8a06382460ea3293a46e9b93b76a243b92f04531fd6bf4ec06fa: Status 404 returned error can't find the container with id 0614fc056e0e8a06382460ea3293a46e9b93b76a243b92f04531fd6bf4ec06fa
Sep 10 21:25:36.045546 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.045513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" event={"ID":"c6e5d478-c1df-443b-a257-77df79edb95a","Type":"ContainerStarted","Data":"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"}
Sep 10 21:25:36.045698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.045576    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:36.045698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.045600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" event={"ID":"c6e5d478-c1df-443b-a257-77df79edb95a","Type":"ContainerStarted","Data":"1d010ba23821e412debedf74bc3b52891f9e496ad6fd3807ca8028f1fa3a6424"}
Sep 10 21:25:36.046702 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.046682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" event={"ID":"d33b2d4a-f1d4-4837-9c12-f277da7a4eda","Type":"ContainerStarted","Data":"0614fc056e0e8a06382460ea3293a46e9b93b76a243b92f04531fd6bf4ec06fa"}
Sep 10 21:25:36.048317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.048294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" event={"ID":"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2","Type":"ContainerStarted","Data":"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"}
Sep 10 21:25:36.048392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.048336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" event={"ID":"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2","Type":"ContainerStarted","Data":"89f4292d69b556cdcc5181b0fd5a6251fc47d4c4e538d8099312f8793bd275d0"}
Sep 10 21:25:36.048446 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.048409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:36.061954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.061896    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" podStartSLOduration=2.061875956 podStartE2EDuration="2.061875956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:36.060771956 +0000 UTC m=+953.939481022" watchObservedRunningTime="2026-09-10 21:25:36.061875956 +0000 UTC m=+953.940585019"
Sep 10 21:25:36.078034 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:36.077989    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" podStartSLOduration=2.077970606 podStartE2EDuration="2.077970606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:36.075232357 +0000 UTC m=+953.953941421" watchObservedRunningTime="2026-09-10 21:25:36.077970606 +0000 UTC m=+953.956679670"
Sep 10 21:25:37.055579 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:37.055542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" event={"ID":"d33b2d4a-f1d4-4837-9c12-f277da7a4eda","Type":"ContainerStarted","Data":"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"}
Sep 10 21:25:37.056045 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:37.055758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:37.072509 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:37.072449    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" podStartSLOduration=3.60523171 podStartE2EDuration="4.07242703s" totalImagesPullingTime="467.19532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:37.071605391 +0000 UTC m=+954.950314453" watchObservedRunningTime="2026-09-10 21:25:37.07242703 +0000 UTC m=+954.951136092"
Sep 10 21:25:43.067360 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:43.067265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:25:43.620119 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:43.620088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:43.620119 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:43.620124    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:43.632633 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:43.632608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:44.106550 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:44.106520    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:25:47.057620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:47.057585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:25:47.058089 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:47.057676    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:25:52.249199 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.249165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:25:52.255577 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.255551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.257833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.257808    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-fbe4cf\""
Sep 10 21:25:52.257954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.257884    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-fbe4cf-dockercfg-b2bgd\""
Sep 10 21:25:52.262249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.262223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:25:52.391426 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.391386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.391570 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.391488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k687c\" (UniqueName: \"kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.492914 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.492882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.493313 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.493261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k687c\" (UniqueName: \"kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.493676 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.493657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.501130 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.501083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k687c\" (UniqueName: \"kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c\") pod \"payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") " pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.568309 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.568262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:52.842650 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:52.842629    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:25:52.844570 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:25:52.844537    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee685bc6_e042_49f3_87cf_03a738b23716.slice/crio-b43b24e28d5812123698e7addd26d5c45a60cfe9cd6913a278ed8ef2a7e41b41 WatchSource:0}: Error finding container b43b24e28d5812123698e7addd26d5c45a60cfe9cd6913a278ed8ef2a7e41b41: Status 404 returned error can't find the container with id b43b24e28d5812123698e7addd26d5c45a60cfe9cd6913a278ed8ef2a7e41b41
Sep 10 21:25:53.163395 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:53.163363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" event={"ID":"ee685bc6-e042-49f3-87cf-03a738b23716","Type":"ContainerStarted","Data":"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"}
Sep 10 21:25:53.163771 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:53.163422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" event={"ID":"ee685bc6-e042-49f3-87cf-03a738b23716","Type":"ContainerStarted","Data":"b43b24e28d5812123698e7addd26d5c45a60cfe9cd6913a278ed8ef2a7e41b41"}
Sep 10 21:25:53.163771 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:53.163451    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:25:53.180838 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:25:53.180797    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" podStartSLOduration=1.180782029 podStartE2EDuration="1.180782029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:53.178143874 +0000 UTC m=+971.056852952" watchObservedRunningTime="2026-09-10 21:25:53.180782029 +0000 UTC m=+971.059491090"
Sep 10 21:26:04.173112 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.173081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:26:04.569976 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.569885    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:26:04.570136 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.570072    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" podUID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" containerName="maas-api" containerID="cri-o://5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3" gracePeriod=30
Sep 10 21:26:04.634411 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.634379    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:26:04.634671 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.634644    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" podUID="c6e5d478-c1df-443b-a257-77df79edb95a" containerName="payload-processing" containerID="cri-o://8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212" gracePeriod=30
Sep 10 21:26:04.640779 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.640753    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:26:04.641031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.641006    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" podUID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" containerName="payload-pre-processing" containerID="cri-o://d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46" gracePeriod=30
Sep 10 21:26:04.904973 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.904937    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:26:04.905617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.905582    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") pod \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") "
Sep 10 21:26:04.905755 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.905638    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca\") pod \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") "
Sep 10 21:26:04.905871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.905765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs\") pod \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") "
Sep 10 21:26:04.905871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.905808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtxgl\" (UniqueName: \"kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl\") pod \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\" (UID: \"d33b2d4a-f1d4-4837-9c12-f277da7a4eda\") "
Sep 10 21:26:04.906180 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.906157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d33b2d4a-f1d4-4837-9c12-f277da7a4eda" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:04.908489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.908468    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d33b2d4a-f1d4-4837-9c12-f277da7a4eda" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:04.908622 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.908597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl" (OuterVolumeSpecName: "kube-api-access-wtxgl") pod "d33b2d4a-f1d4-4837-9c12-f277da7a4eda" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda"). InnerVolumeSpecName "kube-api-access-wtxgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:04.910404 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.909725    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:26:04.910404 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.910389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d33b2d4a-f1d4-4837-9c12-f277da7a4eda" (UID: "d33b2d4a-f1d4-4837-9c12-f277da7a4eda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:04.984541 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:04.984515    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:26:05.006729 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.006698    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.006729 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.006728    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtxgl\" (UniqueName: \"kubernetes.io/projected/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-kube-api-access-wtxgl\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.006900 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.006741    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.006900 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.006754    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33b2d4a-f1d4-4837-9c12-f277da7a4eda-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.108088 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkf69\" (UniqueName: \"kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69\") pod \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") "
Sep 10 21:26:05.108088 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pntw\" (UniqueName: \"kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw\") pod \"c6e5d478-c1df-443b-a257-77df79edb95a\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") "
Sep 10 21:26:05.108314 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108153    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume\") pod \"c6e5d478-c1df-443b-a257-77df79edb95a\" (UID: \"c6e5d478-c1df-443b-a257-77df79edb95a\") "
Sep 10 21:26:05.108314 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108176    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume\") pod \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\" (UID: \"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2\") "
Sep 10 21:26:05.108526 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6e5d478-c1df-443b-a257-77df79edb95a" (UID: "c6e5d478-c1df-443b-a257-77df79edb95a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:05.108600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.108571    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" (UID: "b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:05.110097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.110071    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69" (OuterVolumeSpecName: "kube-api-access-qkf69") pod "b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" (UID: "b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2"). InnerVolumeSpecName "kube-api-access-qkf69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:05.110097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.110091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw" (OuterVolumeSpecName: "kube-api-access-6pntw") pod "c6e5d478-c1df-443b-a257-77df79edb95a" (UID: "c6e5d478-c1df-443b-a257-77df79edb95a"). InnerVolumeSpecName "kube-api-access-6pntw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:05.209641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.209614    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6e5d478-c1df-443b-a257-77df79edb95a-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.209641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.209635    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.209641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.209645    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkf69\" (UniqueName: \"kubernetes.io/projected/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2-kube-api-access-qkf69\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.210023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.209655    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pntw\" (UniqueName: \"kubernetes.io/projected/c6e5d478-c1df-443b-a257-77df79edb95a-kube-api-access-6pntw\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:05.227149 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.227120    2580 generic.go:356] "Generic (PLEG): container finished" podID="c6e5d478-c1df-443b-a257-77df79edb95a" containerID="8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212" exitCode=0
Sep 10 21:26:05.227302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.227158    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"
Sep 10 21:26:05.227302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.227207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" event={"ID":"c6e5d478-c1df-443b-a257-77df79edb95a","Type":"ContainerDied","Data":"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"}
Sep 10 21:26:05.227302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.227293    2580 scope.go:117] "RemoveContainer" containerID="8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"
Sep 10 21:26:05.227468 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.227265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm" event={"ID":"c6e5d478-c1df-443b-a257-77df79edb95a","Type":"ContainerDied","Data":"1d010ba23821e412debedf74bc3b52891f9e496ad6fd3807ca8028f1fa3a6424"}
Sep 10 21:26:05.228600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.228581    2580 generic.go:356] "Generic (PLEG): container finished" podID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" containerID="5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3" exitCode=0
Sep 10 21:26:05.228699 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.228637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"
Sep 10 21:26:05.228699 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.228668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" event={"ID":"d33b2d4a-f1d4-4837-9c12-f277da7a4eda","Type":"ContainerDied","Data":"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"}
Sep 10 21:26:05.228804 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.228713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7" event={"ID":"d33b2d4a-f1d4-4837-9c12-f277da7a4eda","Type":"ContainerDied","Data":"0614fc056e0e8a06382460ea3293a46e9b93b76a243b92f04531fd6bf4ec06fa"}
Sep 10 21:26:05.230252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.230226    2580 generic.go:356] "Generic (PLEG): container finished" podID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" containerID="d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46" exitCode=0
Sep 10 21:26:05.230384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.230308    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"
Sep 10 21:26:05.230461 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.230320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" event={"ID":"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2","Type":"ContainerDied","Data":"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"}
Sep 10 21:26:05.230512 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.230479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg" event={"ID":"b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2","Type":"ContainerDied","Data":"89f4292d69b556cdcc5181b0fd5a6251fc47d4c4e538d8099312f8793bd275d0"}
Sep 10 21:26:05.236603 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.236571    2580 scope.go:117] "RemoveContainer" containerID="8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"
Sep 10 21:26:05.237331 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:26:05.237226    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212\": container with ID starting with 8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212 not found: ID does not exist" containerID="8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"
Sep 10 21:26:05.237331 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.237261    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212"} err="failed to get container status \"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212\": rpc error: code = NotFound desc = could not find container \"8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212\": container with ID starting with 8cf473b9ad460c82c9b76db6f51df3bf45d6cfbc0ca84fed8817c9d2b4687212 not found: ID does not exist"
Sep 10 21:26:05.237331 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.237307    2580 scope.go:117] "RemoveContainer" containerID="5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"
Sep 10 21:26:05.246075 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.246057    2580 scope.go:117] "RemoveContainer" containerID="5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"
Sep 10 21:26:05.246328 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:26:05.246307    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3\": container with ID starting with 5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3 not found: ID does not exist" containerID="5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"
Sep 10 21:26:05.246411 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.246339    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3"} err="failed to get container status \"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3\": rpc error: code = NotFound desc = could not find container \"5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3\": container with ID starting with 5a72f0e6f31485f70ebc1c4c2f6dd5dec6619bc4fc6822c955794a1e8e24adf3 not found: ID does not exist"
Sep 10 21:26:05.246411 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.246361    2580 scope.go:117] "RemoveContainer" containerID="d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"
Sep 10 21:26:05.254154 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.254127    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:26:05.256480 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.256463    2580 scope.go:117] "RemoveContainer" containerID="d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"
Sep 10 21:26:05.257050 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:26:05.257024    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46\": container with ID starting with d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46 not found: ID does not exist" containerID="d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"
Sep 10 21:26:05.257148 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.257055    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46"} err="failed to get container status \"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46\": rpc error: code = NotFound desc = could not find container \"d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46\": container with ID starting with d185c79aa8e2af3595ca08cfbfcc9ae4f8ef9e2a5f96e416bd5d3e6a12d9bf46 not found: ID does not exist"
Sep 10 21:26:05.259356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.259336    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fd195260-6fbbbc747c-jn4wm"]
Sep 10 21:26:05.268165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.268143    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:26:05.272625 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.272608    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fd195260-f49c49878-kj7rg"]
Sep 10 21:26:05.283224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.283205    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:26:05.290091 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:05.290071    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fd195260-84fdd4656f-vs9j7"]
Sep 10 21:26:06.611391 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:06.611358    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" path="/var/lib/kubelet/pods/b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2/volumes"
Sep 10 21:26:06.611746 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:06.611730    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6e5d478-c1df-443b-a257-77df79edb95a" path="/var/lib/kubelet/pods/c6e5d478-c1df-443b-a257-77df79edb95a/volumes"
Sep 10 21:26:06.612061 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:06.612049    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" path="/var/lib/kubelet/pods/d33b2d4a-f1d4-4837-9c12-f277da7a4eda/volumes"
Sep 10 21:26:43.328925 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:43.328893    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:26:44.600104 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"]
Sep 10 21:26:44.600501 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600461    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" containerName="payload-pre-processing"
Sep 10 21:26:44.600501 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600473    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" containerName="payload-pre-processing"
Sep 10 21:26:44.600501 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600481    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6e5d478-c1df-443b-a257-77df79edb95a" containerName="payload-processing"
Sep 10 21:26:44.600501 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600487    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6e5d478-c1df-443b-a257-77df79edb95a" containerName="payload-processing"
Sep 10 21:26:44.600637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600509    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" containerName="maas-api"
Sep 10 21:26:44.600637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600515    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" containerName="maas-api"
Sep 10 21:26:44.600637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600578    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33b2d4a-f1d4-4837-9c12-f277da7a4eda" containerName="maas-api"
Sep 10 21:26:44.600637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600585    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8edd45a-dd7b-46f9-bb91-0dbbb87eecb2" containerName="payload-pre-processing"
Sep 10 21:26:44.600637 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.600592    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6e5d478-c1df-443b-a257-77df79edb95a" containerName="payload-processing"
Sep 10 21:26:44.605036 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.605020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:44.607992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.607970    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:26:44.608127 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.607975    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:26:44.615661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.615637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"]
Sep 10 21:26:44.768657 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.768625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh\" (UID: \"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:44.869426 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.869354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh\" (UID: \"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:44.869936 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.869918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh\" (UID: \"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:44.916368 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:44.916343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:45.188636 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:45.188614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"]
Sep 10 21:26:45.190761 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:26:45.190734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f663e8_9ee1_4e8f_8ecf_5268b5ece45d.slice/crio-b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427 WatchSource:0}: Error finding container b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427: Status 404 returned error can't find the container with id b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427
Sep 10 21:26:45.450136 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:45.450063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh" event={"ID":"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d","Type":"ContainerStarted","Data":"71048240a14f7e475fd54b93655455ccd1a4940dad226ebe439a028c02573f79"}
Sep 10 21:26:45.450253 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:45.450157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh" event={"ID":"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d","Type":"ContainerStarted","Data":"b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427"}
Sep 10 21:26:45.467439 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:45.467392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh" podStartSLOduration=1.467373038 podStartE2EDuration="1.467373038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:26:45.464473547 +0000 UTC m=+1023.343182609" watchObservedRunningTime="2026-09-10 21:26:45.467373038 +0000 UTC m=+1023.346082102"
Sep 10 21:26:46.457698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:46.457661    2580 generic.go:356] "Generic (PLEG): container finished" podID="00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" containerID="71048240a14f7e475fd54b93655455ccd1a4940dad226ebe439a028c02573f79" exitCode=0
Sep 10 21:26:46.458171 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:46.457706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh" event={"ID":"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d","Type":"ContainerDied","Data":"71048240a14f7e475fd54b93655455ccd1a4940dad226ebe439a028c02573f79"}
Sep 10 21:26:47.596406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:47.596383    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:47.789016 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:47.788943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca\") pod \"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d\" (UID: \"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d\") "
Sep 10 21:26:47.789218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:47.789196    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" (UID: "00f663e8-9ee1-4e8f-8ecf-5268b5ece45d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:47.890202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:47.890175    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d-maas-api-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:48.470364 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:48.470338    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"
Sep 10 21:26:48.470533 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:48.470331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh" event={"ID":"00f663e8-9ee1-4e8f-8ecf-5268b5ece45d","Type":"ContainerDied","Data":"b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427"}
Sep 10 21:26:48.470533 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:48.470458    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4239fdfc074a560bfd088f4ed935d4b1f81160a6fe527fe2ff4470fa4e9e427"
Sep 10 21:26:55.196732 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.196699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:26:55.197300 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.196942    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" podUID="ee685bc6-e042-49f3-87cf-03a738b23716" containerName="payload-processing" containerID="cri-o://22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486" gracePeriod=30
Sep 10 21:26:55.412753 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.412721    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"]
Sep 10 21:26:55.415983 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.415956    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-fbe4cf-g9skh"]
Sep 10 21:26:55.452038 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.451982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:26:55.507561 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.507530    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee685bc6-e042-49f3-87cf-03a738b23716" containerID="22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486" exitCode=0
Sep 10 21:26:55.507693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.507581    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" event={"ID":"ee685bc6-e042-49f3-87cf-03a738b23716","Type":"ContainerDied","Data":"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"}
Sep 10 21:26:55.507693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.507590    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"
Sep 10 21:26:55.507693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.507619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x" event={"ID":"ee685bc6-e042-49f3-87cf-03a738b23716","Type":"ContainerDied","Data":"b43b24e28d5812123698e7addd26d5c45a60cfe9cd6913a278ed8ef2a7e41b41"}
Sep 10 21:26:55.507693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.507640    2580 scope.go:117] "RemoveContainer" containerID="22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"
Sep 10 21:26:55.516079 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.516061    2580 scope.go:117] "RemoveContainer" containerID="22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"
Sep 10 21:26:55.516378 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:26:55.516360    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486\": container with ID starting with 22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486 not found: ID does not exist" containerID="22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"
Sep 10 21:26:55.516441 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.516385    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486"} err="failed to get container status \"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486\": rpc error: code = NotFound desc = could not find container \"22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486\": container with ID starting with 22426a9b54aa5c3456d5564a134ca186b6e3c43971604dcbe21ee28203a8f486 not found: ID does not exist"
Sep 10 21:26:55.556897 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.556874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume\") pod \"ee685bc6-e042-49f3-87cf-03a738b23716\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") "
Sep 10 21:26:55.557008 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.556955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k687c\" (UniqueName: \"kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c\") pod \"ee685bc6-e042-49f3-87cf-03a738b23716\" (UID: \"ee685bc6-e042-49f3-87cf-03a738b23716\") "
Sep 10 21:26:55.557353 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.557330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee685bc6-e042-49f3-87cf-03a738b23716" (UID: "ee685bc6-e042-49f3-87cf-03a738b23716"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:55.559019 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.559001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c" (OuterVolumeSpecName: "kube-api-access-k687c") pod "ee685bc6-e042-49f3-87cf-03a738b23716" (UID: "ee685bc6-e042-49f3-87cf-03a738b23716"). InnerVolumeSpecName "kube-api-access-k687c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:55.657788 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.657758    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k687c\" (UniqueName: \"kubernetes.io/projected/ee685bc6-e042-49f3-87cf-03a738b23716-kube-api-access-k687c\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.657889 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.657791    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee685bc6-e042-49f3-87cf-03a738b23716-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.836919 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.836885    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:26:55.837888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:55.837863    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-fbe4cf-555dc4d949-lll6x"]
Sep 10 21:26:56.610847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:56.610813    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" path="/var/lib/kubelet/pods/00f663e8-9ee1-4e8f-8ecf-5268b5ece45d/volumes"
Sep 10 21:26:56.611201 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:56.611168    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee685bc6-e042-49f3-87cf-03a738b23716" path="/var/lib/kubelet/pods/ee685bc6-e042-49f3-87cf-03a738b23716/volumes"
Sep 10 21:26:58.348360 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.348323    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" podUID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" containerName="main" containerID="cri-o://e9daddaf9ee8aef3e34e2cabefecb9b8c87f6ae5e8c27b7576c6ec240815ef75" gracePeriod=45
Sep 10 21:26:58.527011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.526979    2580 generic.go:356] "Generic (PLEG): container finished" podID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" containerID="e9daddaf9ee8aef3e34e2cabefecb9b8c87f6ae5e8c27b7576c6ec240815ef75" exitCode=0
Sep 10 21:26:58.527147 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.527045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" event={"ID":"30d0516f-9e29-4487-8b2b-b01481cfa3c7","Type":"ContainerDied","Data":"e9daddaf9ee8aef3e34e2cabefecb9b8c87f6ae5e8c27b7576c6ec240815ef75"}
Sep 10 21:26:58.589232 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.589208    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:26:58.686906 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.686881    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.686929    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.686982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687028    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687204 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687051    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687204 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687082    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glzmx\" (UniqueName: \"kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx\") pod \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\" (UID: \"30d0516f-9e29-4487-8b2b-b01481cfa3c7\") "
Sep 10 21:26:58.687204 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:58.687388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home" (OuterVolumeSpecName: "home") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:58.687388 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:58.687495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687422    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-home\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:58.687495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687436    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tmp-dir\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:58.687495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.687446    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-model-cache\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:58.689261 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.689234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm" (OuterVolumeSpecName: "dshm") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:58.689261 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.689246    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:58.689425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.689255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx" (OuterVolumeSpecName: "kube-api-access-glzmx") pod "30d0516f-9e29-4487-8b2b-b01481cfa3c7" (UID: "30d0516f-9e29-4487-8b2b-b01481cfa3c7"). InnerVolumeSpecName "kube-api-access-glzmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:58.788869 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.788841    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30d0516f-9e29-4487-8b2b-b01481cfa3c7-tls-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:58.788869 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.788865    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30d0516f-9e29-4487-8b2b-b01481cfa3c7-dshm\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:58.789038 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:58.788875    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glzmx\" (UniqueName: \"kubernetes.io/projected/30d0516f-9e29-4487-8b2b-b01481cfa3c7-kube-api-access-glzmx\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:26:59.533723 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:59.533683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz" event={"ID":"30d0516f-9e29-4487-8b2b-b01481cfa3c7","Type":"ContainerDied","Data":"c89c998438d738e13394ba5ed09abf4ba627c6b756e0b3279aee4c12c968e727"}
Sep 10 21:26:59.533723 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:59.533719    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"
Sep 10 21:26:59.534172 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:59.533746    2580 scope.go:117] "RemoveContainer" containerID="e9daddaf9ee8aef3e34e2cabefecb9b8c87f6ae5e8c27b7576c6ec240815ef75"
Sep 10 21:26:59.558787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:59.558765    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:26:59.563986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:26:59.563966    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-788af409-kserve-6bc4fc4974-cz4lz"]
Sep 10 21:27:00.611543 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:00.611513    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" path="/var/lib/kubelet/pods/30d0516f-9e29-4487-8b2b-b01481cfa3c7/volumes"
Sep 10 21:27:45.277919 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.277885    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278263    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" containerName="main"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278290    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" containerName="main"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278297    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" containerName="revoke-keys"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278303    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" containerName="revoke-keys"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278322    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee685bc6-e042-49f3-87cf-03a738b23716" containerName="payload-processing"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278329    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee685bc6-e042-49f3-87cf-03a738b23716" containerName="payload-processing"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278390    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee685bc6-e042-49f3-87cf-03a738b23716" containerName="payload-processing"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278399    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f663e8-9ee1-4e8f-8ecf-5268b5ece45d" containerName="revoke-keys"
Sep 10 21:27:45.278436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.278407    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d0516f-9e29-4487-8b2b-b01481cfa3c7" containerName="main"
Sep 10 21:27:45.282694 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.282672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.285217 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.285197    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:27:45.286417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.286396    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:27:45.286559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.286513    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:27:45.286694 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.286675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:27:45.286872 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.286846    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-76320447-serving-cert\""
Sep 10 21:27:45.296879 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.296853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:27:45.399084 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.399057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.399208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.399088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46l96\" (UniqueName: \"kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.399208 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.399126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.399309 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.399234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.500351 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.500319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.500508 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.500389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.500508 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.500444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.500508 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.500473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46l96\" (UniqueName: \"kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.501097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.501076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.502798 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.502777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.502915 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.502826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.508077 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.508059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46l96\" (UniqueName: \"kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96\") pod \"maas-api-e2e-mt-76320447-75b59497f-x6czc\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.594069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.594022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:45.870701 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.870607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:27:45.878862 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:27:45.878811    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085c4452_f44c_44cd_9803_0c96814d3462.slice/crio-494c51143107d6a5fb113b8bca1ebccad1b23c1d4dfd8f1fb6c30df478bcc726 WatchSource:0}: Error finding container 494c51143107d6a5fb113b8bca1ebccad1b23c1d4dfd8f1fb6c30df478bcc726: Status 404 returned error can't find the container with id 494c51143107d6a5fb113b8bca1ebccad1b23c1d4dfd8f1fb6c30df478bcc726
Sep 10 21:27:45.879868 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:45.879850    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:27:46.102822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.102789    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:27:46.107589 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.107572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.111104 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.111068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-76320447-dockercfg-wdx5x\""
Sep 10 21:27:46.111233 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.111194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-76320447\""
Sep 10 21:27:46.118965 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.118943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:27:46.205920 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.205888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.206111 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.205935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcwcs\" (UniqueName: \"kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.306787 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.306763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.307159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.306803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcwcs\" (UniqueName: \"kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.307373 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.307355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.314681 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.314661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcwcs\" (UniqueName: \"kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs\") pod \"payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") " pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.419606 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.419577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.700847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.700820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:27:46.702726 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:27:46.702692    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fc1d50_9ffc_4b58_8670_5c4cf14d3300.slice/crio-7d155964fcf131c37f6f27fb4ef5fbcfec1bd6706e2093285eae2373082a586f WatchSource:0}: Error finding container 7d155964fcf131c37f6f27fb4ef5fbcfec1bd6706e2093285eae2373082a586f: Status 404 returned error can't find the container with id 7d155964fcf131c37f6f27fb4ef5fbcfec1bd6706e2093285eae2373082a586f
Sep 10 21:27:46.773679 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.773653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" event={"ID":"085c4452-f44c-44cd-9803-0c96814d3462","Type":"ContainerStarted","Data":"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"}
Sep 10 21:27:46.773798 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.773702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" event={"ID":"085c4452-f44c-44cd-9803-0c96814d3462","Type":"ContainerStarted","Data":"494c51143107d6a5fb113b8bca1ebccad1b23c1d4dfd8f1fb6c30df478bcc726"}
Sep 10 21:27:46.773798 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.773754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:46.775212 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.775189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" event={"ID":"57fc1d50-9ffc-4b58-8670-5c4cf14d3300","Type":"ContainerStarted","Data":"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"}
Sep 10 21:27:46.775341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.775233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" event={"ID":"57fc1d50-9ffc-4b58-8670-5c4cf14d3300","Type":"ContainerStarted","Data":"7d155964fcf131c37f6f27fb4ef5fbcfec1bd6706e2093285eae2373082a586f"}
Sep 10 21:27:46.775341 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.775299    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:27:46.794628 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.794585    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" podStartSLOduration=1.388765395 podStartE2EDuration="1.794571238s" totalImagesPullingTime="405.805843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:46.792612265 +0000 UTC m=+1084.671321343" watchObservedRunningTime="2026-09-10 21:27:46.794571238 +0000 UTC m=+1084.673280297"
Sep 10 21:27:46.811558 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:46.811511    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" podStartSLOduration=0.811495854 podStartE2EDuration="811.495854ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:46.810042374 +0000 UTC m=+1084.688751437" watchObservedRunningTime="2026-09-10 21:27:46.811495854 +0000 UTC m=+1084.690204916"
Sep 10 21:27:49.247800 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:49.247769    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:27:50.243031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.243002    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"]
Sep 10 21:27:50.246611 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.246593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:50.248675 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.248657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:27:50.289750 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.289721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"]
Sep 10 21:27:50.345348 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.345319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc\" (UID: \"f58ec8f3-9305-46ca-a562-7da0b24191df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:50.446215 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.446190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc\" (UID: \"f58ec8f3-9305-46ca-a562-7da0b24191df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:50.446822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.446803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc\" (UID: \"f58ec8f3-9305-46ca-a562-7da0b24191df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:50.557186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.557115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:50.830602 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:50.830565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"]
Sep 10 21:27:50.832415 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:27:50.832387    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58ec8f3_9305_46ca_a562_7da0b24191df.slice/crio-c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899 WatchSource:0}: Error finding container c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899: Status 404 returned error can't find the container with id c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899
Sep 10 21:27:51.802715 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:51.802642    2580 generic.go:356] "Generic (PLEG): container finished" podID="f58ec8f3-9305-46ca-a562-7da0b24191df" containerID="d4825c730ed7eef555bfe55fdf81ebe745fb44ed61af46f4f0456c0433b89045" exitCode=0
Sep 10 21:27:51.803041 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:51.802713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc" event={"ID":"f58ec8f3-9305-46ca-a562-7da0b24191df","Type":"ContainerDied","Data":"d4825c730ed7eef555bfe55fdf81ebe745fb44ed61af46f4f0456c0433b89045"}
Sep 10 21:27:51.803041 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:51.802750    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc" event={"ID":"f58ec8f3-9305-46ca-a562-7da0b24191df","Type":"ContainerStarted","Data":"c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899"}
Sep 10 21:27:52.786231 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:52.786201    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:27:52.945886 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:52.945865    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:53.069401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.069329    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca\") pod \"f58ec8f3-9305-46ca-a562-7da0b24191df\" (UID: \"f58ec8f3-9305-46ca-a562-7da0b24191df\") "
Sep 10 21:27:53.069711 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.069691    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f58ec8f3-9305-46ca-a562-7da0b24191df" (UID: "f58ec8f3-9305-46ca-a562-7da0b24191df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:27:53.170458 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.170425    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58ec8f3-9305-46ca-a562-7da0b24191df-maas-api-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:27:53.813819 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.813789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc" event={"ID":"f58ec8f3-9305-46ca-a562-7da0b24191df","Type":"ContainerDied","Data":"c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899"}
Sep 10 21:27:53.813819 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.813823    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"
Sep 10 21:27:53.814032 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:53.813834    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9338ca4b77313ee795304d2321f8b44bcaddaba80949a2b3d3db220da6bd899"
Sep 10 21:27:57.783234 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:27:57.783208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:28:01.182735 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:01.182706    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"]
Sep 10 21:28:01.185946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:01.185922    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ec041-wq4zc"]
Sep 10 21:28:02.611778 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:02.611745    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58ec8f3-9305-46ca-a562-7da0b24191df" path="/var/lib/kubelet/pods/f58ec8f3-9305-46ca-a562-7da0b24191df/volumes"
Sep 10 21:28:04.268148 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.268111    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" podUID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" containerName="main" containerID="cri-o://fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc" gracePeriod=45
Sep 10 21:28:04.524503 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.524450    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:28:04.674844 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.674818    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f5c4\" (UniqueName: \"kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.674995 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.674880    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.674995 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.674950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.674995 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.674974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.675116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.675116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675083    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache\") pod \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\" (UID: \"1bfce600-d66f-400e-8cd8-98c33f56dfeb\") "
Sep 10 21:28:04.675213 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675162    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home" (OuterVolumeSpecName: "home") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:04.675257 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675229    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:04.675392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:04.675488 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675476    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-home\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.675530 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675492    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tmp-dir\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.675530 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.675503    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-model-cache\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.677213 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.677189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm" (OuterVolumeSpecName: "dshm") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:04.677405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.677380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:04.677405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.677396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4" (OuterVolumeSpecName: "kube-api-access-4f5c4") pod "1bfce600-d66f-400e-8cd8-98c33f56dfeb" (UID: "1bfce600-d66f-400e-8cd8-98c33f56dfeb"). InnerVolumeSpecName "kube-api-access-4f5c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:04.776641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.776579    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfce600-d66f-400e-8cd8-98c33f56dfeb-tls-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.776641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.776601    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfce600-d66f-400e-8cd8-98c33f56dfeb-dshm\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.776641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.776611    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f5c4\" (UniqueName: \"kubernetes.io/projected/1bfce600-d66f-400e-8cd8-98c33f56dfeb-kube-api-access-4f5c4\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:04.870877 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.870852    2580 generic.go:356] "Generic (PLEG): container finished" podID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" containerID="fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc" exitCode=0
Sep 10 21:28:04.870989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.870898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" event={"ID":"1bfce600-d66f-400e-8cd8-98c33f56dfeb","Type":"ContainerDied","Data":"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"}
Sep 10 21:28:04.870989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.870912    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"
Sep 10 21:28:04.870989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.870925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn" event={"ID":"1bfce600-d66f-400e-8cd8-98c33f56dfeb","Type":"ContainerDied","Data":"8d715a214105ba5f9e73902aa5e551858844fc346dd6d1d5866bc6287bda318e"}
Sep 10 21:28:04.870989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.870943    2580 scope.go:117] "RemoveContainer" containerID="fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"
Sep 10 21:28:04.879965 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.879948    2580 scope.go:117] "RemoveContainer" containerID="fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"
Sep 10 21:28:04.880223 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:04.880206    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc\": container with ID starting with fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc not found: ID does not exist" containerID="fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"
Sep 10 21:28:04.880281 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.880231    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc"} err="failed to get container status \"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc\": rpc error: code = NotFound desc = could not find container \"fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc\": container with ID starting with fdfff18c7d81cf8b366604c4ee4c7df03de2de10c3fb0baad94608912615c9bc not found: ID does not exist"
Sep 10 21:28:04.892351 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.892331    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:28:04.895680 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:04.895657    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5ec041/test-model-5ec041-kserve-775fc5b5df-455xn"]
Sep 10 21:28:06.610632 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:06.610598    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" path="/var/lib/kubelet/pods/1bfce600-d66f-400e-8cd8-98c33f56dfeb/volumes"
Sep 10 21:28:08.492037 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492002    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"]
Sep 10 21:28:08.492443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492391    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" containerName="main"
Sep 10 21:28:08.492443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492402    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" containerName="main"
Sep 10 21:28:08.492443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492420    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58ec8f3-9305-46ca-a562-7da0b24191df" containerName="revoke-keys"
Sep 10 21:28:08.492443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492426    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58ec8f3-9305-46ca-a562-7da0b24191df" containerName="revoke-keys"
Sep 10 21:28:08.492588 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492502    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bfce600-d66f-400e-8cd8-98c33f56dfeb" containerName="main"
Sep 10 21:28:08.492588 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.492512    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58ec8f3-9305-46ca-a562-7da0b24191df" containerName="revoke-keys"
Sep 10 21:28:08.497040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.497024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:08.499464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.499441    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:28:08.502808 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.502785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"]
Sep 10 21:28:08.607714 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.607688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k\" (UID: \"84073f56-a8b4-4657-af55-647d6811c438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:08.708320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.708266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k\" (UID: \"84073f56-a8b4-4657-af55-647d6811c438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:08.708801 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.708782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k\" (UID: \"84073f56-a8b4-4657-af55-647d6811c438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:08.810265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:08.810202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:09.087786 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:09.087762    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"]
Sep 10 21:28:09.090181 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:28:09.090153    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84073f56_a8b4_4657_af55_647d6811c438.slice/crio-06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b WatchSource:0}: Error finding container 06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b: Status 404 returned error can't find the container with id 06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b
Sep 10 21:28:09.898679 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:09.898643    2580 generic.go:356] "Generic (PLEG): container finished" podID="84073f56-a8b4-4657-af55-647d6811c438" containerID="579ec60beac2585b52113b896b8763a5d4c0256d6c447589abee200fda108f7b" exitCode=0
Sep 10 21:28:09.899001 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:09.898690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k" event={"ID":"84073f56-a8b4-4657-af55-647d6811c438","Type":"ContainerDied","Data":"579ec60beac2585b52113b896b8763a5d4c0256d6c447589abee200fda108f7b"}
Sep 10 21:28:09.899001 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:09.898737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k" event={"ID":"84073f56-a8b4-4657-af55-647d6811c438","Type":"ContainerStarted","Data":"06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b"}
Sep 10 21:28:11.035455 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.035433    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:11.124501 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.124472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca\") pod \"84073f56-a8b4-4657-af55-647d6811c438\" (UID: \"84073f56-a8b4-4657-af55-647d6811c438\") "
Sep 10 21:28:11.124796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.124776    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84073f56-a8b4-4657-af55-647d6811c438" (UID: "84073f56-a8b4-4657-af55-647d6811c438"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:11.225686 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.225619    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84073f56-a8b4-4657-af55-647d6811c438-maas-api-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:11.911087 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.911055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k" event={"ID":"84073f56-a8b4-4657-af55-647d6811c438","Type":"ContainerDied","Data":"06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b"}
Sep 10 21:28:11.911087 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.911078    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"
Sep 10 21:28:11.911308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:11.911099    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06563edf89193f207e4e98826f00b22472265e22d8885850f10284233cc2649b"
Sep 10 21:28:19.520044 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.520008    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:28:19.520659 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.520250    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" podUID="abb17544-c1e7-4860-9fc5-993e3a222958" containerName="payload-processing" containerID="cri-o://7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4" gracePeriod=30
Sep 10 21:28:19.783626 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.783563    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"]
Sep 10 21:28:19.784183 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.784164    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:28:19.786541 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.786521    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6441b-qnc4k"]
Sep 10 21:28:19.906138 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.906110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfqks\" (UniqueName: \"kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks\") pod \"abb17544-c1e7-4860-9fc5-993e3a222958\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") "
Sep 10 21:28:19.906304 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.906162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume\") pod \"abb17544-c1e7-4860-9fc5-993e3a222958\" (UID: \"abb17544-c1e7-4860-9fc5-993e3a222958\") "
Sep 10 21:28:19.906578 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.906550    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abb17544-c1e7-4860-9fc5-993e3a222958" (UID: "abb17544-c1e7-4860-9fc5-993e3a222958"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:19.908108 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.908083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks" (OuterVolumeSpecName: "kube-api-access-tfqks") pod "abb17544-c1e7-4860-9fc5-993e3a222958" (UID: "abb17544-c1e7-4860-9fc5-993e3a222958"). InnerVolumeSpecName "kube-api-access-tfqks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:19.962323 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.962291    2580 generic.go:356] "Generic (PLEG): container finished" podID="abb17544-c1e7-4860-9fc5-993e3a222958" containerID="7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4" exitCode=0
Sep 10 21:28:19.962445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.962331    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"
Sep 10 21:28:19.962445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.962360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" event={"ID":"abb17544-c1e7-4860-9fc5-993e3a222958","Type":"ContainerDied","Data":"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"}
Sep 10 21:28:19.962445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.962413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6" event={"ID":"abb17544-c1e7-4860-9fc5-993e3a222958","Type":"ContainerDied","Data":"1239828ec61c47b7b49c31e9e0b986900b84d6821203b42c5a655034a0221b61"}
Sep 10 21:28:19.962445 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.962437    2580 scope.go:117] "RemoveContainer" containerID="7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"
Sep 10 21:28:19.971091 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.971074    2580 scope.go:117] "RemoveContainer" containerID="7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"
Sep 10 21:28:19.971372 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:19.971353    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4\": container with ID starting with 7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4 not found: ID does not exist" containerID="7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"
Sep 10 21:28:19.971425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.971379    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4"} err="failed to get container status \"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4\": rpc error: code = NotFound desc = could not find container \"7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4\": container with ID starting with 7148bb7bdd5a35ec73ee7498a517be316b71da407f799d15c217c0a62627a7a4 not found: ID does not exist"
Sep 10 21:28:19.984291 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.984249    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:28:19.988095 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:19.988073    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a6441b-58f65ccd4c-wj5d6"]
Sep 10 21:28:20.007614 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:20.007594    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb17544-c1e7-4860-9fc5-993e3a222958-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.007698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:20.007616    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfqks\" (UniqueName: \"kubernetes.io/projected/abb17544-c1e7-4860-9fc5-993e3a222958-kube-api-access-tfqks\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.611529 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:20.611497    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84073f56-a8b4-4657-af55-647d6811c438" path="/var/lib/kubelet/pods/84073f56-a8b4-4657-af55-647d6811c438/volumes"
Sep 10 21:28:20.612023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:20.612002    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb17544-c1e7-4860-9fc5-993e3a222958" path="/var/lib/kubelet/pods/abb17544-c1e7-4860-9fc5-993e3a222958/volumes"
Sep 10 21:28:27.184219 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.184188    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:28:27.184686 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.184444    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" podUID="085c4452-f44c-44cd-9803-0c96814d3462" containerName="maas-api" containerID="cri-o://5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4" gracePeriod=30
Sep 10 21:28:27.247844 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.247815    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:28:27.248077 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.248050    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" podUID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" containerName="payload-processing" containerID="cri-o://4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce" gracePeriod=30
Sep 10 21:28:27.503198 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.503170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:28:27.514456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.514430    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:28:27.569874 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.569849    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca\") pod \"085c4452-f44c-44cd-9803-0c96814d3462\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") "
Sep 10 21:28:27.570027 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.569939    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs\") pod \"085c4452-f44c-44cd-9803-0c96814d3462\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") "
Sep 10 21:28:27.570112 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46l96\" (UniqueName: \"kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96\") pod \"085c4452-f44c-44cd-9803-0c96814d3462\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") "
Sep 10 21:28:27.570160 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570123    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcwcs\" (UniqueName: \"kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs\") pod \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") "
Sep 10 21:28:27.570244 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570224    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls\") pod \"085c4452-f44c-44cd-9803-0c96814d3462\" (UID: \"085c4452-f44c-44cd-9803-0c96814d3462\") "
Sep 10 21:28:27.570326 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "085c4452-f44c-44cd-9803-0c96814d3462" (UID: "085c4452-f44c-44cd-9803-0c96814d3462"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:27.570396 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume\") pod \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\" (UID: \"57fc1d50-9ffc-4b58-8670-5c4cf14d3300\") "
Sep 10 21:28:27.570703 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570678    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085c4452-f44c-44cd-9803-0c96814d3462-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.570839 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.570743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57fc1d50-9ffc-4b58-8670-5c4cf14d3300" (UID: "57fc1d50-9ffc-4b58-8670-5c4cf14d3300"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:27.572325 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.572258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "085c4452-f44c-44cd-9803-0c96814d3462" (UID: "085c4452-f44c-44cd-9803-0c96814d3462"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:27.572325 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.572297    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs" (OuterVolumeSpecName: "kube-api-access-dcwcs") pod "57fc1d50-9ffc-4b58-8670-5c4cf14d3300" (UID: "57fc1d50-9ffc-4b58-8670-5c4cf14d3300"). InnerVolumeSpecName "kube-api-access-dcwcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:27.572460 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.572304    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "085c4452-f44c-44cd-9803-0c96814d3462" (UID: "085c4452-f44c-44cd-9803-0c96814d3462"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:27.572574 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.572557    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96" (OuterVolumeSpecName: "kube-api-access-46l96") pod "085c4452-f44c-44cd-9803-0c96814d3462" (UID: "085c4452-f44c-44cd-9803-0c96814d3462"). InnerVolumeSpecName "kube-api-access-46l96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:27.671759 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.671730    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.671759 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.671752    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46l96\" (UniqueName: \"kubernetes.io/projected/085c4452-f44c-44cd-9803-0c96814d3462-kube-api-access-46l96\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.671759 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.671763    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcwcs\" (UniqueName: \"kubernetes.io/projected/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-kube-api-access-dcwcs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.671964 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.671771    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085c4452-f44c-44cd-9803-0c96814d3462-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.671964 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:27.671779    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57fc1d50-9ffc-4b58-8670-5c4cf14d3300-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:28:28.011649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.011615    2580 generic.go:356] "Generic (PLEG): container finished" podID="085c4452-f44c-44cd-9803-0c96814d3462" containerID="5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4" exitCode=0
Sep 10 21:28:28.011857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.011679    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"
Sep 10 21:28:28.011857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.011701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" event={"ID":"085c4452-f44c-44cd-9803-0c96814d3462","Type":"ContainerDied","Data":"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"}
Sep 10 21:28:28.011857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.011794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc" event={"ID":"085c4452-f44c-44cd-9803-0c96814d3462","Type":"ContainerDied","Data":"494c51143107d6a5fb113b8bca1ebccad1b23c1d4dfd8f1fb6c30df478bcc726"}
Sep 10 21:28:28.011857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.011823    2580 scope.go:117] "RemoveContainer" containerID="5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"
Sep 10 21:28:28.013416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.013386    2580 generic.go:356] "Generic (PLEG): container finished" podID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" containerID="4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce" exitCode=0
Sep 10 21:28:28.013531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.013428    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"
Sep 10 21:28:28.013531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.013456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" event={"ID":"57fc1d50-9ffc-4b58-8670-5c4cf14d3300","Type":"ContainerDied","Data":"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"}
Sep 10 21:28:28.013531 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.013490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p" event={"ID":"57fc1d50-9ffc-4b58-8670-5c4cf14d3300","Type":"ContainerDied","Data":"7d155964fcf131c37f6f27fb4ef5fbcfec1bd6706e2093285eae2373082a586f"}
Sep 10 21:28:28.022924 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.022908    2580 scope.go:117] "RemoveContainer" containerID="5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"
Sep 10 21:28:28.023239 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:28.023221    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4\": container with ID starting with 5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4 not found: ID does not exist" containerID="5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"
Sep 10 21:28:28.023321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.023247    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4"} err="failed to get container status \"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4\": rpc error: code = NotFound desc = could not find container \"5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4\": container with ID starting with 5d92af294ebb2c3175ebbed23fc995d8cd6066de542f1678f19e6fbc4af58dd4 not found: ID does not exist"
Sep 10 21:28:28.023321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.023263    2580 scope.go:117] "RemoveContainer" containerID="4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"
Sep 10 21:28:28.031072 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.031057    2580 scope.go:117] "RemoveContainer" containerID="4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"
Sep 10 21:28:28.031339 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:28.031319    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce\": container with ID starting with 4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce not found: ID does not exist" containerID="4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"
Sep 10 21:28:28.031385 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.031349    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce"} err="failed to get container status \"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce\": rpc error: code = NotFound desc = could not find container \"4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce\": container with ID starting with 4fec4cae83a93904ad8fd7f4dd41efc4fd6c2fbff4f95b9689569781846f24ce not found: ID does not exist"
Sep 10 21:28:28.034734 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.034713    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:28:28.042178 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.042158    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-76320447-75b59497f-x6czc"]
Sep 10 21:28:28.053799 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.053773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:28:28.059397 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.059375    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-76320447-76748c8bb5-h4m8p"]
Sep 10 21:28:28.612606 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.612577    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="085c4452-f44c-44cd-9803-0c96814d3462" path="/var/lib/kubelet/pods/085c4452-f44c-44cd-9803-0c96814d3462/volumes"
Sep 10 21:28:28.612973 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:28.612923    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" path="/var/lib/kubelet/pods/57fc1d50-9ffc-4b58-8670-5c4cf14d3300/volumes"
Sep 10 21:28:31.274976 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.274940    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:28:31.275515 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275496    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="085c4452-f44c-44cd-9803-0c96814d3462" containerName="maas-api"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275518    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="085c4452-f44c-44cd-9803-0c96814d3462" containerName="maas-api"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275531    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84073f56-a8b4-4657-af55-647d6811c438" containerName="revoke-keys"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275539    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84073f56-a8b4-4657-af55-647d6811c438" containerName="revoke-keys"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275557    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" containerName="payload-processing"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" containerName="payload-processing"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275584    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb17544-c1e7-4860-9fc5-993e3a222958" containerName="payload-processing"
Sep 10 21:28:31.275616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275592    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb17544-c1e7-4860-9fc5-993e3a222958" containerName="payload-processing"
Sep 10 21:28:31.275946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275702    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="085c4452-f44c-44cd-9803-0c96814d3462" containerName="maas-api"
Sep 10 21:28:31.275946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275716    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57fc1d50-9ffc-4b58-8670-5c4cf14d3300" containerName="payload-processing"
Sep 10 21:28:31.275946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275728    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb17544-c1e7-4860-9fc5-993e3a222958" containerName="payload-processing"
Sep 10 21:28:31.275946 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.275738    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="84073f56-a8b4-4657-af55-647d6811c438" containerName="revoke-keys"
Sep 10 21:28:31.280312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.280289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.282796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.282776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-36b938-serving-cert\""
Sep 10 21:28:31.283009 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.282994    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:28:31.283749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.283727    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:28:31.283847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.283794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:28:31.283847 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.283795    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:28:31.293091 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.293068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:28:31.303757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.303726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.303849 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.303793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhnk\" (UniqueName: \"kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.303849 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.303823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.303967 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.303947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.404759 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.404731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.404874 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.404769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.404979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.404954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.405078 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.405053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhnk\" (UniqueName: \"kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.405192 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:31.405118    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-serving-cert: secret "maas-api-e2e-shared-a-w3-36b938-serving-cert" not found
Sep 10 21:28:31.405350 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:31.405214    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls podName:8f03dd60-205c-41ba-8509-9e6b986c9f67 nodeName:}" failed. No retries permitted until 2026-09-10 21:28:31.905192665 +0000 UTC m=+1129.783901721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls") pod "maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" (UID: "8f03dd60-205c-41ba-8509-9e6b986c9f67") : secret "maas-api-e2e-shared-a-w3-36b938-serving-cert" not found
Sep 10 21:28:31.405512 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.405472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.407772 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.407751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.415961 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.415939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhnk\" (UniqueName: \"kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.910382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.910351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:31.912559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:31.912540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:32.012231 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.012201    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:28:32.015935 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.015921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.018495 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.018474    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:28:32.018600 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.018556    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-36b938-dockercfg-ld5s2\""
Sep 10 21:28:32.019044 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.019026    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-36b938\""
Sep 10 21:28:32.021205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.021189    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:28:32.030027 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.030005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:28:32.112507 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.112468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.112669 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.112648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rht\" (UniqueName: \"kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.192312 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.192218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:32.213683 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.213653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rht\" (UniqueName: \"kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.213846 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.213710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.214468 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.214447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.225226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.225204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rht\" (UniqueName: \"kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht\") pod \"payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.327048 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.327016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:32.464018 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.463928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:28:32.466304 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:28:32.466266    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f03dd60_205c_41ba_8509_9e6b986c9f67.slice/crio-b95b7bea42b7ccf97364ef37db093315995eb5355c2516b369c564466b8d45a0 WatchSource:0}: Error finding container b95b7bea42b7ccf97364ef37db093315995eb5355c2516b369c564466b8d45a0: Status 404 returned error can't find the container with id b95b7bea42b7ccf97364ef37db093315995eb5355c2516b369c564466b8d45a0
Sep 10 21:28:32.601941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:32.601920    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:28:32.604000 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:28:32.603977    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0116ad7b_2753_4006_8c6b_9713c5210762.slice/crio-fb1b9ef14c5583daefd2e98ea4da1dfb777632483758f959abf68bb2a8eeb727 WatchSource:0}: Error finding container fb1b9ef14c5583daefd2e98ea4da1dfb777632483758f959abf68bb2a8eeb727: Status 404 returned error can't find the container with id fb1b9ef14c5583daefd2e98ea4da1dfb777632483758f959abf68bb2a8eeb727
Sep 10 21:28:33.044117 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.044082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" event={"ID":"8f03dd60-205c-41ba-8509-9e6b986c9f67","Type":"ContainerStarted","Data":"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"}
Sep 10 21:28:33.044319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.044142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" event={"ID":"8f03dd60-205c-41ba-8509-9e6b986c9f67","Type":"ContainerStarted","Data":"b95b7bea42b7ccf97364ef37db093315995eb5355c2516b369c564466b8d45a0"}
Sep 10 21:28:33.044319 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.044188    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:33.045687 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.045661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" event={"ID":"0116ad7b-2753-4006-8c6b-9713c5210762","Type":"ContainerStarted","Data":"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"}
Sep 10 21:28:33.045817 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.045706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" event={"ID":"0116ad7b-2753-4006-8c6b-9713c5210762","Type":"ContainerStarted","Data":"fb1b9ef14c5583daefd2e98ea4da1dfb777632483758f959abf68bb2a8eeb727"}
Sep 10 21:28:33.045817 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.045777    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:33.063263 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.063214    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" podStartSLOduration=1.669972859 podStartE2EDuration="2.063195166s" totalImagesPullingTime="393.222307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:33.062025806 +0000 UTC m=+1130.940734868" watchObservedRunningTime="2026-09-10 21:28:33.063195166 +0000 UTC m=+1130.941904230"
Sep 10 21:28:33.081266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:33.081224    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" podStartSLOduration=2.081211899 podStartE2EDuration="2.081211899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:33.078984167 +0000 UTC m=+1130.957693229" watchObservedRunningTime="2026-09-10 21:28:33.081211899 +0000 UTC m=+1130.959921001"
Sep 10 21:28:39.056021 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:39.055991    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:28:44.052479 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:44.052449    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:28:59.701839 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.701802    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:28:59.706882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.706862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.709583 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.709564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-1f3879-serving-cert\""
Sep 10 21:28:59.721514 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.721495    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:28:59.857627 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.857592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.857795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.857639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.857795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.857725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.857917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.857803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqvfk\" (UniqueName: \"kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.959093 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.959019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqvfk\" (UniqueName: \"kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.959093 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.959055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.959093 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.959092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.959400 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.959145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.959400 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:59.959185    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-serving-cert: secret "maas-api-e2e-shared-b-w3-1f3879-serving-cert" not found
Sep 10 21:28:59.959400 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:28:59.959254    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls podName:e35abf66-4377-496e-ae0d-2007e88e021d nodeName:}" failed. No retries permitted until 2026-09-10 21:29:00.459233453 +0000 UTC m=+1158.337942516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls") pod "maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" (UID: "e35abf66-4377-496e-ae0d-2007e88e021d") : secret "maas-api-e2e-shared-b-w3-1f3879-serving-cert" not found
Sep 10 21:28:59.959888 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.959868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.961775 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.961748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:28:59.967325 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:28:59.967299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqvfk\" (UniqueName: \"kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:00.464229 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.464185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:00.466514 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.466492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:00.587259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.587230    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:29:00.591265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.591244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.600500 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.600481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-1f3879\""
Sep 10 21:29:00.600616 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.600596    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-1f3879-dockercfg-hr6bn\""
Sep 10 21:29:00.612385 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.612365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:29:00.617969 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.617943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:00.665965 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.665919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.666143 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.665978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dgsv\" (UniqueName: \"kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.766938 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.766848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.766938 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.766886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dgsv\" (UniqueName: \"kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.767711 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.767687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.778704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.778678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dgsv\" (UniqueName: \"kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv\") pod \"payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:00.896533 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.896511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:29:00.898325 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:29:00.898296    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35abf66_4377_496e_ae0d_2007e88e021d.slice/crio-119c533b75a21863b9b7d64e7f8f7d15bcd3470c8ad8af8305cd1a892fc6eb98 WatchSource:0}: Error finding container 119c533b75a21863b9b7d64e7f8f7d15bcd3470c8ad8af8305cd1a892fc6eb98: Status 404 returned error can't find the container with id 119c533b75a21863b9b7d64e7f8f7d15bcd3470c8ad8af8305cd1a892fc6eb98
Sep 10 21:29:00.902213 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:00.902180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:01.177620 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:01.177596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:29:01.179631 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:29:01.179607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaefc7b6_48d8_48a7_9c8a_e412d115868e.slice/crio-af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d WatchSource:0}: Error finding container af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d: Status 404 returned error can't find the container with id af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d
Sep 10 21:29:01.193936 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:01.193914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" event={"ID":"eaefc7b6-48d8-48a7-9c8a-e412d115868e","Type":"ContainerStarted","Data":"af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d"}
Sep 10 21:29:01.195258 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:01.195239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" event={"ID":"e35abf66-4377-496e-ae0d-2007e88e021d","Type":"ContainerStarted","Data":"119c533b75a21863b9b7d64e7f8f7d15bcd3470c8ad8af8305cd1a892fc6eb98"}
Sep 10 21:29:02.202460 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.202426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" event={"ID":"eaefc7b6-48d8-48a7-9c8a-e412d115868e","Type":"ContainerStarted","Data":"9a02ac133056c691262693b40ec806c2f5a97366a97c6fa88b88ca120b8c21ed"}
Sep 10 21:29:02.202899 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.202489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:02.203983 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.203960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" event={"ID":"e35abf66-4377-496e-ae0d-2007e88e021d","Type":"ContainerStarted","Data":"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"}
Sep 10 21:29:02.204104 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.204001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:02.223440 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.223400    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" podStartSLOduration=2.223386603 podStartE2EDuration="2.223386603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:02.221740142 +0000 UTC m=+1160.100449205" watchObservedRunningTime="2026-09-10 21:29:02.223386603 +0000 UTC m=+1160.102095666"
Sep 10 21:29:02.241162 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:02.241125    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" podStartSLOduration=2.920924714 podStartE2EDuration="3.241111282s" totalImagesPullingTime="320.186568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:02.239371548 +0000 UTC m=+1160.118080612" watchObservedRunningTime="2026-09-10 21:29:02.241111282 +0000 UTC m=+1160.119820344"
Sep 10 21:29:08.214784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:08.214705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:29:13.212452 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:13.212420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:29:21.780677 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.780645    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:29:21.784473 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.784451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.786828 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.786807    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5e6835-serving-cert\""
Sep 10 21:29:21.796462 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.796441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:29:21.858978 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.858945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.859131 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.858986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.859131 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.859004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb8bh\" (UniqueName: \"kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.859131 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.859081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.959765 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.959734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.959913 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.959808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.959913 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.959862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.959913 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.959885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb8bh\" (UniqueName: \"kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.960056 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:29:21.959918    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-serving-cert: secret "maas-api-e2e-shared-b-w2-5e6835-serving-cert" not found
Sep 10 21:29:21.960056 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:29:21.960005    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls podName:27c178b9-067c-492f-af67-311253d00445 nodeName:}" failed. No retries permitted until 2026-09-10 21:29:22.459983158 +0000 UTC m=+1180.338692215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" (UID: "27c178b9-067c-492f-af67-311253d00445") : secret "maas-api-e2e-shared-b-w2-5e6835-serving-cert" not found
Sep 10 21:29:21.960685 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.960658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.962159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.962134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:21.967829 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:21.967808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb8bh\" (UniqueName: \"kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:22.464716 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.464676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:22.467009 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.466983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:22.619048 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.619008    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:29:22.622820 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.622797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.624770 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.624746    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5e6835\""
Sep 10 21:29:22.624871 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.624796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5e6835-dockercfg-s46mb\""
Sep 10 21:29:22.633203 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.633183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:29:22.667077 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.667048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j27lv\" (UniqueName: \"kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.667202 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.667181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.696195 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.696170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:22.767872 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.767842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j27lv\" (UniqueName: \"kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.767986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.767914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.768608 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.768585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.777296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.777253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j27lv\" (UniqueName: \"kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv\") pod \"payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.934661 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.934632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:22.971155 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:22.971089    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:29:22.974154 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:29:22.974128    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c178b9_067c_492f_af67_311253d00445.slice/crio-1757a25e349d06ec34bbd4ba1f9156b64ad26f584f77b3ae312b4ddead9c20c1 WatchSource:0}: Error finding container 1757a25e349d06ec34bbd4ba1f9156b64ad26f584f77b3ae312b4ddead9c20c1: Status 404 returned error can't find the container with id 1757a25e349d06ec34bbd4ba1f9156b64ad26f584f77b3ae312b4ddead9c20c1
Sep 10 21:29:23.215812 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.215789    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:29:23.217328 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:29:23.217305    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef97bc0f_45b5_479a_b59f_c75a38f4d977.slice/crio-b17e0259282f1a670c39c4fbf4ba3c82e97366046d3e500a61c24905fad29f2b WatchSource:0}: Error finding container b17e0259282f1a670c39c4fbf4ba3c82e97366046d3e500a61c24905fad29f2b: Status 404 returned error can't find the container with id b17e0259282f1a670c39c4fbf4ba3c82e97366046d3e500a61c24905fad29f2b
Sep 10 21:29:23.322145 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.322120    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" event={"ID":"27c178b9-067c-492f-af67-311253d00445","Type":"ContainerStarted","Data":"1757a25e349d06ec34bbd4ba1f9156b64ad26f584f77b3ae312b4ddead9c20c1"}
Sep 10 21:29:23.323761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.323734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" event={"ID":"ef97bc0f-45b5-479a-b59f-c75a38f4d977","Type":"ContainerStarted","Data":"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"}
Sep 10 21:29:23.323873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.323781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" event={"ID":"ef97bc0f-45b5-479a-b59f-c75a38f4d977","Type":"ContainerStarted","Data":"b17e0259282f1a670c39c4fbf4ba3c82e97366046d3e500a61c24905fad29f2b"}
Sep 10 21:29:23.323873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.323857    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:23.342879 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:23.342832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" podStartSLOduration=1.342790284 podStartE2EDuration="1.342790284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:23.340973078 +0000 UTC m=+1181.219682141" watchObservedRunningTime="2026-09-10 21:29:23.342790284 +0000 UTC m=+1181.221499347"
Sep 10 21:29:24.329911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:24.329873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" event={"ID":"27c178b9-067c-492f-af67-311253d00445","Type":"ContainerStarted","Data":"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"}
Sep 10 21:29:24.330302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:24.330216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:24.353444 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:24.353392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" podStartSLOduration=3.028600533 podStartE2EDuration="3.35337648s" totalImagesPullingTime="324.775947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:24.351932604 +0000 UTC m=+1182.230641666" watchObservedRunningTime="2026-09-10 21:29:24.35337648 +0000 UTC m=+1182.232085542"
Sep 10 21:29:30.340436 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:30.340402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:29:34.332476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:34.332440    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:29:42.650384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:42.650348    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:29:42.650384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:42.650362    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:29:59.070742 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.070709    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:29:59.074937 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.074915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.079926 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.079904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7ce89dd6-serving-cert\""
Sep 10 21:29:59.090729 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.090702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:29:59.117676 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.117654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.117785 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.117687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.117785 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.117711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6j5\" (UniqueName: \"kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.117911 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.117884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.218425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.218399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.218563 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.218477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.218563 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.218512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.218563 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.218537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6j5\" (UniqueName: \"kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.218736 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:29:59.218606    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-serving-cert: secret "maas-api-e2e-mt-7ce89dd6-serving-cert" not found
Sep 10 21:29:59.218736 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:29:59.218674    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls podName:8ec03759-7efa-415d-8803-fbce4cb64031 nodeName:}" failed. No retries permitted until 2026-09-10 21:29:59.718657508 +0000 UTC m=+1217.597366565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls") pod "maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" (UID: "8ec03759-7efa-415d-8803-fbce4cb64031") : secret "maas-api-e2e-mt-7ce89dd6-serving-cert" not found
Sep 10 21:29:59.219159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.219142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.220811 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.220793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.229476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.229458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6j5\" (UniqueName: \"kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.724023 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.723937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.726513 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.726483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") pod \"maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:29:59.986010 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:29:59.985922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:30:00.260761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:00.260734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:30:00.262040 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:30:00.262008    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec03759_7efa_415d_8803_fbce4cb64031.slice/crio-dc2c4730e10a434eac22b14e94983518d1c7bb91dc7b770d701a9be8e3a361b4 WatchSource:0}: Error finding container dc2c4730e10a434eac22b14e94983518d1c7bb91dc7b770d701a9be8e3a361b4: Status 404 returned error can't find the container with id dc2c4730e10a434eac22b14e94983518d1c7bb91dc7b770d701a9be8e3a361b4
Sep 10 21:30:00.520237 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:00.520160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" event={"ID":"8ec03759-7efa-415d-8803-fbce4cb64031","Type":"ContainerStarted","Data":"dc2c4730e10a434eac22b14e94983518d1c7bb91dc7b770d701a9be8e3a361b4"}
Sep 10 21:30:01.527165 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:01.527126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" event={"ID":"8ec03759-7efa-415d-8803-fbce4cb64031","Type":"ContainerStarted","Data":"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"}
Sep 10 21:30:01.527617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:01.527219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:30:01.549992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:01.549939    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" podStartSLOduration=2.152560972 podStartE2EDuration="2.549926576s" totalImagesPullingTime="397.365604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:01.548322835 +0000 UTC m=+1219.427031898" watchObservedRunningTime="2026-09-10 21:30:01.549926576 +0000 UTC m=+1219.428635637"
Sep 10 21:30:07.538151 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:30:07.538121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:31:33.105581 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.105545    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:31:33.106133 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.105791    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" podUID="8ec03759-7efa-415d-8803-fbce4cb64031" containerName="maas-api" containerID="cri-o://b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80" gracePeriod=30
Sep 10 21:31:33.364592 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.364531    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:31:33.456324 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.456294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs\") pod \"8ec03759-7efa-415d-8803-fbce4cb64031\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") "
Sep 10 21:31:33.456483 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.456339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql6j5\" (UniqueName: \"kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5\") pod \"8ec03759-7efa-415d-8803-fbce4cb64031\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") "
Sep 10 21:31:33.456483 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.456398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") pod \"8ec03759-7efa-415d-8803-fbce4cb64031\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") "
Sep 10 21:31:33.456483 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.456440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca\") pod \"8ec03759-7efa-415d-8803-fbce4cb64031\" (UID: \"8ec03759-7efa-415d-8803-fbce4cb64031\") "
Sep 10 21:31:33.457092 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.457049    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ec03759-7efa-415d-8803-fbce4cb64031" (UID: "8ec03759-7efa-415d-8803-fbce4cb64031"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:31:33.458740 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.458717    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8ec03759-7efa-415d-8803-fbce4cb64031" (UID: "8ec03759-7efa-415d-8803-fbce4cb64031"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:31:33.458740 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.458729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5" (OuterVolumeSpecName: "kube-api-access-ql6j5") pod "8ec03759-7efa-415d-8803-fbce4cb64031" (UID: "8ec03759-7efa-415d-8803-fbce4cb64031"). InnerVolumeSpecName "kube-api-access-ql6j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:31:33.459039 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.459019    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ec03759-7efa-415d-8803-fbce4cb64031" (UID: "8ec03759-7efa-415d-8803-fbce4cb64031"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:31:33.557988 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.557942    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.557988 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.557984    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql6j5\" (UniqueName: \"kubernetes.io/projected/8ec03759-7efa-415d-8803-fbce4cb64031-kube-api-access-ql6j5\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.558212 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.557999    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ec03759-7efa-415d-8803-fbce4cb64031-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.558212 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:33.558015    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec03759-7efa-415d-8803-fbce4cb64031-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:31:34.019284 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.019236    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ec03759-7efa-415d-8803-fbce4cb64031" containerID="b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80" exitCode=0
Sep 10 21:31:34.019464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.019298    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"
Sep 10 21:31:34.019464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.019324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" event={"ID":"8ec03759-7efa-415d-8803-fbce4cb64031","Type":"ContainerDied","Data":"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"}
Sep 10 21:31:34.019464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.019380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq" event={"ID":"8ec03759-7efa-415d-8803-fbce4cb64031","Type":"ContainerDied","Data":"dc2c4730e10a434eac22b14e94983518d1c7bb91dc7b770d701a9be8e3a361b4"}
Sep 10 21:31:34.019464 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.019409    2580 scope.go:117] "RemoveContainer" containerID="b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"
Sep 10 21:31:34.030017 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.029998    2580 scope.go:117] "RemoveContainer" containerID="b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"
Sep 10 21:31:34.030320 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:31:34.030298    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80\": container with ID starting with b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80 not found: ID does not exist" containerID="b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"
Sep 10 21:31:34.030397 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.030328    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80"} err="failed to get container status \"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80\": rpc error: code = NotFound desc = could not find container \"b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80\": container with ID starting with b36f5f76289e7b58c5e612b4c28277cb4ac9f71de6bbdb02d9a267c09b8aca80 not found: ID does not exist"
Sep 10 21:31:34.044421 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.044398    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:31:34.047986 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.047966    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7ce89dd6-79f7d5c976-fcgnq"]
Sep 10 21:31:34.611651 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:34.611615    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec03759-7efa-415d-8803-fbce4cb64031" path="/var/lib/kubelet/pods/8ec03759-7efa-415d-8803-fbce4cb64031/volumes"
Sep 10 21:31:42.749433 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:31:42.749404    2580 scope.go:117] "RemoveContainer" containerID="5a7d034bb89ee9531cdfa64fbd5c518503919842fd03056bdf6d3c2642e43448"
Sep 10 21:33:09.468392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.468359    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:33:09.469012 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.468995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec03759-7efa-415d-8803-fbce4cb64031" containerName="maas-api"
Sep 10 21:33:09.469056 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.469015    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec03759-7efa-415d-8803-fbce4cb64031" containerName="maas-api"
Sep 10 21:33:09.469140 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.469129    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec03759-7efa-415d-8803-fbce4cb64031" containerName="maas-api"
Sep 10 21:33:09.472860 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.472839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.475169 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.475142    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-77a957-serving-cert\""
Sep 10 21:33:09.482933 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.482911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:33:09.489879 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.489856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.489981 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.489898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nppp\" (UniqueName: \"kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.490097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.490076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.490141 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.490118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.590857 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.590827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.590994 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.590860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.590994 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.590911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.590994 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.590943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nppp\" (UniqueName: \"kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.590994 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:33:09.590973    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-serving-cert: secret "maas-api-e2e-api-a-77a957-serving-cert" not found
Sep 10 21:33:09.591201 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:33:09.591055    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls podName:345c0b31-8369-4272-b315-4bd6e23d43be nodeName:}" failed. No retries permitted until 2026-09-10 21:33:10.09103586 +0000 UTC m=+1407.969744903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls") pod "maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" (UID: "345c0b31-8369-4272-b315-4bd6e23d43be") : secret "maas-api-e2e-api-a-77a957-serving-cert" not found
Sep 10 21:33:09.591652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.591630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.593321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.593301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:09.599791 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:09.599769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nppp\" (UniqueName: \"kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:10.096524 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:10.096484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:10.099045 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:10.099019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") pod \"maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:10.384979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:10.384951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:10.662422 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:10.662394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:33:10.664420 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:33:10.664393    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345c0b31_8369_4272_b315_4bd6e23d43be.slice/crio-77b14765fc02e2bd3db2af60710690da6c32fa0c57f5e6c9bd157b5e6f10fcf8 WatchSource:0}: Error finding container 77b14765fc02e2bd3db2af60710690da6c32fa0c57f5e6c9bd157b5e6f10fcf8: Status 404 returned error can't find the container with id 77b14765fc02e2bd3db2af60710690da6c32fa0c57f5e6c9bd157b5e6f10fcf8
Sep 10 21:33:10.666000 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:10.665982    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:33:11.542200 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:11.542124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" event={"ID":"345c0b31-8369-4272-b315-4bd6e23d43be","Type":"ContainerStarted","Data":"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"}
Sep 10 21:33:11.542200 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:11.542178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" event={"ID":"345c0b31-8369-4272-b315-4bd6e23d43be","Type":"ContainerStarted","Data":"77b14765fc02e2bd3db2af60710690da6c32fa0c57f5e6c9bd157b5e6f10fcf8"}
Sep 10 21:33:11.542399 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:11.542205    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:11.560401 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:11.560355    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" podStartSLOduration=2.073807457 podStartE2EDuration="2.56033991s" totalImagesPullingTime="486.532453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:11.558315844 +0000 UTC m=+1409.437024907" watchObservedRunningTime="2026-09-10 21:33:11.56033991 +0000 UTC m=+1409.439048971"
Sep 10 21:33:17.553673 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:17.553640    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:33:39.282214 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.282137    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:33:39.286011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.285993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.288363 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.288343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c71bd0-dockercfg-7xbgd\""
Sep 10 21:33:39.288363 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.288351    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c71bd0\""
Sep 10 21:33:39.299317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.299295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:33:39.326971 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.326944    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:33:39.330535 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.330517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.340417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.340394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:33:39.363641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.363613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.363739 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.363645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqbjr\" (UniqueName: \"kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.363785 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.363761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbfj2\" (UniqueName: \"kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.363819 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.363797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.464474 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.464442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqbjr\" (UniqueName: \"kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.464630 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.464530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbfj2\" (UniqueName: \"kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.464630 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.464552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.464630 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.464594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.465293 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.465251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.465346 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.465264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.472413 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.472385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqbjr\" (UniqueName: \"kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr\") pod \"payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.472562 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.472540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbfj2\" (UniqueName: \"kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2\") pod \"payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") " pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.596078 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.596015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:39.641011 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.640933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:39.868020 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.867993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:33:39.869046 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:33:39.869020    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703bf5c3_2c36_4ba1_893b_8c85f217fedf.slice/crio-299a275aa2378719d301540bc335d2c4d486d6b800e38cbc30964a11b1752ec0 WatchSource:0}: Error finding container 299a275aa2378719d301540bc335d2c4d486d6b800e38cbc30964a11b1752ec0: Status 404 returned error can't find the container with id 299a275aa2378719d301540bc335d2c4d486d6b800e38cbc30964a11b1752ec0
Sep 10 21:33:39.942603 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:39.942551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:33:39.944966 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:33:39.944944    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a23f267_d249_4dc9_bd4b_bb3525af1178.slice/crio-d7e9b0c875a1ede5cc77cc36bac68747fadc789b3c39003c2aa7c14f78acbdb2 WatchSource:0}: Error finding container d7e9b0c875a1ede5cc77cc36bac68747fadc789b3c39003c2aa7c14f78acbdb2: Status 404 returned error can't find the container with id d7e9b0c875a1ede5cc77cc36bac68747fadc789b3c39003c2aa7c14f78acbdb2
Sep 10 21:33:40.697040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.697007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" event={"ID":"703bf5c3-2c36-4ba1-893b-8c85f217fedf","Type":"ContainerStarted","Data":"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"}
Sep 10 21:33:40.697522 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.697055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" event={"ID":"703bf5c3-2c36-4ba1-893b-8c85f217fedf","Type":"ContainerStarted","Data":"299a275aa2378719d301540bc335d2c4d486d6b800e38cbc30964a11b1752ec0"}
Sep 10 21:33:40.697522 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.697097    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:33:40.698634 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.698615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" event={"ID":"8a23f267-d249-4dc9-bd4b-bb3525af1178","Type":"ContainerStarted","Data":"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"}
Sep 10 21:33:40.698708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.698648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" event={"ID":"8a23f267-d249-4dc9-bd4b-bb3525af1178","Type":"ContainerStarted","Data":"d7e9b0c875a1ede5cc77cc36bac68747fadc789b3c39003c2aa7c14f78acbdb2"}
Sep 10 21:33:40.698764 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.698751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:40.713977 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.713939    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" podStartSLOduration=1.713924343 podStartE2EDuration="1.713924343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:40.713140444 +0000 UTC m=+1438.591849510" watchObservedRunningTime="2026-09-10 21:33:40.713924343 +0000 UTC m=+1438.592633407"
Sep 10 21:33:40.729686 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:40.729639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" podStartSLOduration=1.729627466 podStartE2EDuration="1.729627466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:40.728837532 +0000 UTC m=+1438.607546595" watchObservedRunningTime="2026-09-10 21:33:40.729627466 +0000 UTC m=+1438.608336528"
Sep 10 21:33:42.782101 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:42.782069    2580 scope.go:117] "RemoveContainer" containerID="71048240a14f7e475fd54b93655455ccd1a4940dad226ebe439a028c02573f79"
Sep 10 21:33:51.707249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:51.707219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:33:51.707720 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:33:51.707297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:34:20.344095 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.344062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:34:20.344619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.344285    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" podUID="345c0b31-8369-4272-b315-4bd6e23d43be" containerName="maas-api" containerID="cri-o://eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330" gracePeriod=30
Sep 10 21:34:20.587856 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.587835    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:34:20.637320 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.637294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") pod \"345c0b31-8369-4272-b315-4bd6e23d43be\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") "
Sep 10 21:34:20.637474 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.637351    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs\") pod \"345c0b31-8369-4272-b315-4bd6e23d43be\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") "
Sep 10 21:34:20.637474 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.637379    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca\") pod \"345c0b31-8369-4272-b315-4bd6e23d43be\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") "
Sep 10 21:34:20.637474 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.637402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nppp\" (UniqueName: \"kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp\") pod \"345c0b31-8369-4272-b315-4bd6e23d43be\" (UID: \"345c0b31-8369-4272-b315-4bd6e23d43be\") "
Sep 10 21:34:20.637821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.637794    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "345c0b31-8369-4272-b315-4bd6e23d43be" (UID: "345c0b31-8369-4272-b315-4bd6e23d43be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:20.639467 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.639435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "345c0b31-8369-4272-b315-4bd6e23d43be" (UID: "345c0b31-8369-4272-b315-4bd6e23d43be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:20.639580 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.639475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "345c0b31-8369-4272-b315-4bd6e23d43be" (UID: "345c0b31-8369-4272-b315-4bd6e23d43be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:20.639580 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.639486    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp" (OuterVolumeSpecName: "kube-api-access-6nppp") pod "345c0b31-8369-4272-b315-4bd6e23d43be" (UID: "345c0b31-8369-4272-b315-4bd6e23d43be"). InnerVolumeSpecName "kube-api-access-6nppp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:20.738503 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.738474    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.738503 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.738500    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/345c0b31-8369-4272-b315-4bd6e23d43be-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.738503 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.738508    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/345c0b31-8369-4272-b315-4bd6e23d43be-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.738717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.738516    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nppp\" (UniqueName: \"kubernetes.io/projected/345c0b31-8369-4272-b315-4bd6e23d43be-kube-api-access-6nppp\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.914493 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.914407    2580 generic.go:356] "Generic (PLEG): container finished" podID="345c0b31-8369-4272-b315-4bd6e23d43be" containerID="eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330" exitCode=0
Sep 10 21:34:20.914493 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.914451    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"
Sep 10 21:34:20.914695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.914533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" event={"ID":"345c0b31-8369-4272-b315-4bd6e23d43be","Type":"ContainerDied","Data":"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"}
Sep 10 21:34:20.914695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.914583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr" event={"ID":"345c0b31-8369-4272-b315-4bd6e23d43be","Type":"ContainerDied","Data":"77b14765fc02e2bd3db2af60710690da6c32fa0c57f5e6c9bd157b5e6f10fcf8"}
Sep 10 21:34:20.914695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.914605    2580 scope.go:117] "RemoveContainer" containerID="eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"
Sep 10 21:34:20.923891 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.923863    2580 scope.go:117] "RemoveContainer" containerID="eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"
Sep 10 21:34:20.924158 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:34:20.924141    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330\": container with ID starting with eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330 not found: ID does not exist" containerID="eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"
Sep 10 21:34:20.924216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.924167    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330"} err="failed to get container status \"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330\": rpc error: code = NotFound desc = could not find container \"eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330\": container with ID starting with eeff91811355d6052661d0d14a497241f687aa08693e633402d04cbb3a337330 not found: ID does not exist"
Sep 10 21:34:20.936169 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.936146    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:34:20.942409 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:20.942390    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a957-6f896bb45f-rmcvr"]
Sep 10 21:34:22.611683 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:22.611649    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345c0b31-8369-4272-b315-4bd6e23d43be" path="/var/lib/kubelet/pods/345c0b31-8369-4272-b315-4bd6e23d43be/volumes"
Sep 10 21:34:42.692146 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.692122    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:34:42.695168 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.695151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:34:42.802380 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.802356    2580 scope.go:117] "RemoveContainer" containerID="d4825c730ed7eef555bfe55fdf81ebe745fb44ed61af46f4f0456c0433b89045"
Sep 10 21:34:42.810704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.810687    2580 scope.go:117] "RemoveContainer" containerID="579ec60beac2585b52113b896b8763a5d4c0256d6c447589abee200fda108f7b"
Sep 10 21:34:42.960559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.960476    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:34:42.961092 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.961062    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" podUID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" containerName="payload-processing" containerID="cri-o://0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e" gracePeriod=30
Sep 10 21:34:42.972741 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.971671    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:34:42.972741 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:42.971914    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" podUID="8a23f267-d249-4dc9-bd4b-bb3525af1178" containerName="payload-pre-processing" containerID="cri-o://21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b" gracePeriod=30
Sep 10 21:34:43.247252 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.247230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:34:43.250182 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.250157    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:34:43.340046 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340023    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume\") pod \"8a23f267-d249-4dc9-bd4b-bb3525af1178\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") "
Sep 10 21:34:43.340216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340064    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqbjr\" (UniqueName: \"kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr\") pod \"8a23f267-d249-4dc9-bd4b-bb3525af1178\" (UID: \"8a23f267-d249-4dc9-bd4b-bb3525af1178\") "
Sep 10 21:34:43.340216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbfj2\" (UniqueName: \"kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2\") pod \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") "
Sep 10 21:34:43.340216 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340177    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume\") pod \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\" (UID: \"703bf5c3-2c36-4ba1-893b-8c85f217fedf\") "
Sep 10 21:34:43.340549 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a23f267-d249-4dc9-bd4b-bb3525af1178" (UID: "8a23f267-d249-4dc9-bd4b-bb3525af1178"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:43.340674 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.340577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "703bf5c3-2c36-4ba1-893b-8c85f217fedf" (UID: "703bf5c3-2c36-4ba1-893b-8c85f217fedf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:43.342175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.342151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr" (OuterVolumeSpecName: "kube-api-access-bqbjr") pod "8a23f267-d249-4dc9-bd4b-bb3525af1178" (UID: "8a23f267-d249-4dc9-bd4b-bb3525af1178"). InnerVolumeSpecName "kube-api-access-bqbjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:43.342261 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.342245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2" (OuterVolumeSpecName: "kube-api-access-gbfj2") pod "703bf5c3-2c36-4ba1-893b-8c85f217fedf" (UID: "703bf5c3-2c36-4ba1-893b-8c85f217fedf"). InnerVolumeSpecName "kube-api-access-gbfj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:43.441717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.441684    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703bf5c3-2c36-4ba1-893b-8c85f217fedf-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.441717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.441715    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a23f267-d249-4dc9-bd4b-bb3525af1178-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.441717 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.441725    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqbjr\" (UniqueName: \"kubernetes.io/projected/8a23f267-d249-4dc9-bd4b-bb3525af1178-kube-api-access-bqbjr\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.441902 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:43.441734    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbfj2\" (UniqueName: \"kubernetes.io/projected/703bf5c3-2c36-4ba1-893b-8c85f217fedf-kube-api-access-gbfj2\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:34:44.043240 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.043204    2580 generic.go:356] "Generic (PLEG): container finished" podID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" containerID="0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e" exitCode=0
Sep 10 21:34:44.043700 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.043240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" event={"ID":"703bf5c3-2c36-4ba1-893b-8c85f217fedf","Type":"ContainerDied","Data":"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"}
Sep 10 21:34:44.043700 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.043325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2" event={"ID":"703bf5c3-2c36-4ba1-893b-8c85f217fedf","Type":"ContainerDied","Data":"299a275aa2378719d301540bc335d2c4d486d6b800e38cbc30964a11b1752ec0"}
Sep 10 21:34:44.043700 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.043250    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"
Sep 10 21:34:44.043700 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.043353    2580 scope.go:117] "RemoveContainer" containerID="0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"
Sep 10 21:34:44.044900 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.044877    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a23f267-d249-4dc9-bd4b-bb3525af1178" containerID="21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b" exitCode=0
Sep 10 21:34:44.044976 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.044922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" event={"ID":"8a23f267-d249-4dc9-bd4b-bb3525af1178","Type":"ContainerDied","Data":"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"}
Sep 10 21:34:44.044976 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.044929    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"
Sep 10 21:34:44.044976 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.044962    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2" event={"ID":"8a23f267-d249-4dc9-bd4b-bb3525af1178","Type":"ContainerDied","Data":"d7e9b0c875a1ede5cc77cc36bac68747fadc789b3c39003c2aa7c14f78acbdb2"}
Sep 10 21:34:44.052844 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.052820    2580 scope.go:117] "RemoveContainer" containerID="0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"
Sep 10 21:34:44.053247 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:34:44.053226    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e\": container with ID starting with 0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e not found: ID does not exist" containerID="0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"
Sep 10 21:34:44.053432 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.053256    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e"} err="failed to get container status \"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e\": rpc error: code = NotFound desc = could not find container \"0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e\": container with ID starting with 0fa92e85d10ab4d5fbb8e27cebb1bb47b604c8038661a6f6688e9843280d036e not found: ID does not exist"
Sep 10 21:34:44.053432 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.053300    2580 scope.go:117] "RemoveContainer" containerID="21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"
Sep 10 21:34:44.062959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.062938    2580 scope.go:117] "RemoveContainer" containerID="21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"
Sep 10 21:34:44.063226 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:34:44.063204    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b\": container with ID starting with 21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b not found: ID does not exist" containerID="21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"
Sep 10 21:34:44.063321 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.063237    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b"} err="failed to get container status \"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b\": rpc error: code = NotFound desc = could not find container \"21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b\": container with ID starting with 21805b007df3175f6fe3ff85c798a17dd909e337d9e1efc6fd96eb137907d73b not found: ID does not exist"
Sep 10 21:34:44.072860 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.072839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:34:44.079215 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.079187    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c71bd0-575d657f9f-gnkz2"]
Sep 10 21:34:44.091854 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.091835    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:34:44.096185 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.096167    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c71bd0-54db8787f4-qk5w2"]
Sep 10 21:34:44.611953 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.611920    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" path="/var/lib/kubelet/pods/703bf5c3-2c36-4ba1-893b-8c85f217fedf/volumes"
Sep 10 21:34:44.612390 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:34:44.612370    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a23f267-d249-4dc9-bd4b-bb3525af1178" path="/var/lib/kubelet/pods/8a23f267-d249-4dc9-bd4b-bb3525af1178/volumes"
Sep 10 21:35:16.239296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239197    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239612    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" containerName="payload-processing"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239623    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" containerName="payload-processing"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345c0b31-8369-4272-b315-4bd6e23d43be" containerName="maas-api"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239648    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="345c0b31-8369-4272-b315-4bd6e23d43be" containerName="maas-api"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239658    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a23f267-d249-4dc9-bd4b-bb3525af1178" containerName="payload-pre-processing"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a23f267-d249-4dc9-bd4b-bb3525af1178" containerName="payload-pre-processing"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239732    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="345c0b31-8369-4272-b315-4bd6e23d43be" containerName="maas-api"
Sep 10 21:35:16.239737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239741    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="703bf5c3-2c36-4ba1-893b-8c85f217fedf" containerName="payload-processing"
Sep 10 21:35:16.240016 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.239748    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a23f267-d249-4dc9-bd4b-bb3525af1178" containerName="payload-pre-processing"
Sep 10 21:35:16.242809 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.242785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.245542 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.245519    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c8d758-dockercfg-bx5w9\""
Sep 10 21:35:16.245542 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.245526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c8d758\""
Sep 10 21:35:16.252704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.252682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:35:16.327736 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.327710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57mz9\" (UniqueName: \"kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.327891 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.327755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.428962 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.428930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.429099 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.429029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57mz9\" (UniqueName: \"kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.429678 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.429657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.443263 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.443237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57mz9\" (UniqueName: \"kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9\") pod \"payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.557335 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.557263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:16.839259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:16.839236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:35:16.841123 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:35:16.841098    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a805821_fe2c_4121_8616_f02e47669f66.slice/crio-0d6bd660f8ee8d1842efee59308864078bf43b35c55e691fe50a4a9a739b1fee WatchSource:0}: Error finding container 0d6bd660f8ee8d1842efee59308864078bf43b35c55e691fe50a4a9a739b1fee: Status 404 returned error can't find the container with id 0d6bd660f8ee8d1842efee59308864078bf43b35c55e691fe50a4a9a739b1fee
Sep 10 21:35:17.234992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:17.234960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" event={"ID":"8a805821-fe2c-4121-8616-f02e47669f66","Type":"ContainerStarted","Data":"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"}
Sep 10 21:35:17.234992 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:17.235008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" event={"ID":"8a805821-fe2c-4121-8616-f02e47669f66","Type":"ContainerStarted","Data":"0d6bd660f8ee8d1842efee59308864078bf43b35c55e691fe50a4a9a739b1fee"}
Sep 10 21:35:17.235249 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:17.235099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:35:18.388750 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.388697    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" podStartSLOduration=2.388678238 podStartE2EDuration="2.388678238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:17.254900509 +0000 UTC m=+1535.133609584" watchObservedRunningTime="2026-09-10 21:35:18.388678238 +0000 UTC m=+1536.267387346"
Sep 10 21:35:18.390657 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.389995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:35:18.390657 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.390217    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" podUID="27c178b9-067c-492f-af67-311253d00445" containerName="maas-api" containerID="cri-o://19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed" gracePeriod=30
Sep 10 21:35:18.462107 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.462079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:35:18.462522 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.462349    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" podUID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" containerName="payload-pre-processing" containerID="cri-o://29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021" gracePeriod=30
Sep 10 21:35:18.718032 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.718006    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:35:18.747964 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.747941    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs\") pod \"27c178b9-067c-492f-af67-311253d00445\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") "
Sep 10 21:35:18.748089 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.748007    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") pod \"27c178b9-067c-492f-af67-311253d00445\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") "
Sep 10 21:35:18.748089 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.748071    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca\") pod \"27c178b9-067c-492f-af67-311253d00445\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") "
Sep 10 21:35:18.748205 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.748129    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb8bh\" (UniqueName: \"kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh\") pod \"27c178b9-067c-492f-af67-311253d00445\" (UID: \"27c178b9-067c-492f-af67-311253d00445\") "
Sep 10 21:35:18.749069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.749037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27c178b9-067c-492f-af67-311253d00445" (UID: "27c178b9-067c-492f-af67-311253d00445"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:18.750418 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.750396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27c178b9-067c-492f-af67-311253d00445" (UID: "27c178b9-067c-492f-af67-311253d00445"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:18.750586 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.750562    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "27c178b9-067c-492f-af67-311253d00445" (UID: "27c178b9-067c-492f-af67-311253d00445"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:35:18.750667 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.750589    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:35:18.750722 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.750683    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh" (OuterVolumeSpecName: "kube-api-access-kb8bh") pod "27c178b9-067c-492f-af67-311253d00445" (UID: "27c178b9-067c-492f-af67-311253d00445"). InnerVolumeSpecName "kube-api-access-kb8bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:18.849218 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849185    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j27lv\" (UniqueName: \"kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv\") pod \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") "
Sep 10 21:35:18.849422 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume\") pod \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\" (UID: \"ef97bc0f-45b5-479a-b59f-c75a38f4d977\") "
Sep 10 21:35:18.849784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849637    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb8bh\" (UniqueName: \"kubernetes.io/projected/27c178b9-067c-492f-af67-311253d00445-kube-api-access-kb8bh\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.849784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849661    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.849784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849676    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27c178b9-067c-492f-af67-311253d00445-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.849784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.849690    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27c178b9-067c-492f-af67-311253d00445-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.850082 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.850043    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef97bc0f-45b5-479a-b59f-c75a38f4d977" (UID: "ef97bc0f-45b5-479a-b59f-c75a38f4d977"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:18.854030 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.853983    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv" (OuterVolumeSpecName: "kube-api-access-j27lv") pod "ef97bc0f-45b5-479a-b59f-c75a38f4d977" (UID: "ef97bc0f-45b5-479a-b59f-c75a38f4d977"). InnerVolumeSpecName "kube-api-access-j27lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:18.950476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.950410    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j27lv\" (UniqueName: \"kubernetes.io/projected/ef97bc0f-45b5-479a-b59f-c75a38f4d977-kube-api-access-j27lv\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.950476 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:18.950439    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef97bc0f-45b5-479a-b59f-c75a38f4d977-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:35:19.246821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.246746    2580 generic.go:356] "Generic (PLEG): container finished" podID="27c178b9-067c-492f-af67-311253d00445" containerID="19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed" exitCode=0
Sep 10 21:35:19.246821 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.246786    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"
Sep 10 21:35:19.247069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.246834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" event={"ID":"27c178b9-067c-492f-af67-311253d00445","Type":"ContainerDied","Data":"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"}
Sep 10 21:35:19.247069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.246893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd" event={"ID":"27c178b9-067c-492f-af67-311253d00445","Type":"ContainerDied","Data":"1757a25e349d06ec34bbd4ba1f9156b64ad26f584f77b3ae312b4ddead9c20c1"}
Sep 10 21:35:19.247069 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.246923    2580 scope.go:117] "RemoveContainer" containerID="19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"
Sep 10 21:35:19.248147 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.248126    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" containerID="29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021" exitCode=0
Sep 10 21:35:19.248226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.248172    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"
Sep 10 21:35:19.248226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.248177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" event={"ID":"ef97bc0f-45b5-479a-b59f-c75a38f4d977","Type":"ContainerDied","Data":"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"}
Sep 10 21:35:19.248226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.248213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f" event={"ID":"ef97bc0f-45b5-479a-b59f-c75a38f4d977","Type":"ContainerDied","Data":"b17e0259282f1a670c39c4fbf4ba3c82e97366046d3e500a61c24905fad29f2b"}
Sep 10 21:35:19.260833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.260218    2580 scope.go:117] "RemoveContainer" containerID="19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"
Sep 10 21:35:19.260833 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:35:19.260654    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed\": container with ID starting with 19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed not found: ID does not exist" containerID="19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"
Sep 10 21:35:19.260833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.260688    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed"} err="failed to get container status \"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed\": rpc error: code = NotFound desc = could not find container \"19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed\": container with ID starting with 19aced0a08cef53c3558a74474b7050702e1540d98c7f04748885ba660fefaed not found: ID does not exist"
Sep 10 21:35:19.260833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.260713    2580 scope.go:117] "RemoveContainer" containerID="29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"
Sep 10 21:35:19.273301 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.273285    2580 scope.go:117] "RemoveContainer" containerID="29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"
Sep 10 21:35:19.273559 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:35:19.273543    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021\": container with ID starting with 29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021 not found: ID does not exist" containerID="29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"
Sep 10 21:35:19.273604 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.273564    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021"} err="failed to get container status \"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021\": rpc error: code = NotFound desc = could not find container \"29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021\": container with ID starting with 29d6c360f7a328c4c25a1c7c22b7694f83456fba6cfff9609b761249f53ec021 not found: ID does not exist"
Sep 10 21:35:19.280431 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.280410    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:35:19.284386 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.284363    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e6835-6446c669c-p6nvd"]
Sep 10 21:35:19.294372 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.294352    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:35:19.300619 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:19.300598    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5e6835-644477fdb6-hkn2f"]
Sep 10 21:35:20.612116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:20.612078    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c178b9-067c-492f-af67-311253d00445" path="/var/lib/kubelet/pods/27c178b9-067c-492f-af67-311253d00445/volumes"
Sep 10 21:35:20.612632 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:20.612610    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" path="/var/lib/kubelet/pods/ef97bc0f-45b5-479a-b59f-c75a38f4d977/volumes"
Sep 10 21:35:28.242781 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:35:28.242752    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:36:40.742000 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:40.741913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h_8a805821-fe2c-4121-8616-f02e47669f66/payload-processing/0.log"
Sep 10 21:36:45.670861 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.670828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:36:45.671356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671326    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c178b9-067c-492f-af67-311253d00445" containerName="maas-api"
Sep 10 21:36:45.671356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671340    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c178b9-067c-492f-af67-311253d00445" containerName="maas-api"
Sep 10 21:36:45.671356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671355    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" containerName="payload-pre-processing"
Sep 10 21:36:45.671356 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671360    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" containerName="payload-pre-processing"
Sep 10 21:36:45.671605 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671434    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef97bc0f-45b5-479a-b59f-c75a38f4d977" containerName="payload-pre-processing"
Sep 10 21:36:45.671605 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.671445    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c178b9-067c-492f-af67-311253d00445" containerName="maas-api"
Sep 10 21:36:45.674463 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.674443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.676641 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.676618    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-389568-serving-cert\""
Sep 10 21:36:45.683557 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.683533    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:36:45.816160 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.816122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk89p\" (UniqueName: \"kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.816406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.816173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.816406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.816233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.816406 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.816374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.917536 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.917507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk89p\" (UniqueName: \"kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.917690 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.917548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.917690 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.917584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.917690 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.917636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.917826 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:36:45.917762    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-serving-cert: secret "maas-api-e2e-ipp-cleanup-389568-serving-cert" not found
Sep 10 21:36:45.917863 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:36:45.917832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls podName:7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75 nodeName:}" failed. No retries permitted until 2026-09-10 21:36:46.417809661 +0000 UTC m=+1624.296518714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" (UID: "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75") : secret "maas-api-e2e-ipp-cleanup-389568-serving-cert" not found
Sep 10 21:36:45.918302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.918258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.919954 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.919938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:45.927603 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:45.927551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk89p\" (UniqueName: \"kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:46.300308 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.300210    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:36:46.303859 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.303838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.307597 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.307576    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-389568\""
Sep 10 21:36:46.307824 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.307801    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-389568-dockercfg-kccz2\""
Sep 10 21:36:46.314587 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.314565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:36:46.423061 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.423030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:46.423211 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.423128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6tb6\" (UniqueName: \"kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.423211 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.423163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.425384 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.425360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:46.524170 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.524135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.524338 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.524298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6tb6\" (UniqueName: \"kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.524914 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.524895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.531790 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.531772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6tb6\" (UniqueName: \"kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6\") pod \"payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.587525 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.587469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:46.616302 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.616264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:46.861364 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.861288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:36:46.864854 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:36:46.864830    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb719f8_3cd4_4b8c_a5ac_7417dd8d1e75.slice/crio-ad62564a1afe4cf52ef2155ad7c103c9e5a474051a1867f17ad6857e7ab370ec WatchSource:0}: Error finding container ad62564a1afe4cf52ef2155ad7c103c9e5a474051a1867f17ad6857e7ab370ec: Status 404 returned error can't find the container with id ad62564a1afe4cf52ef2155ad7c103c9e5a474051a1867f17ad6857e7ab370ec
Sep 10 21:36:46.895967 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:46.895944    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:36:46.897790 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:36:46.897757    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f33d12e_b354_4834_9217_f449b01fd12c.slice/crio-a25d745a60e3eab86a40bd49b35ad167f71d7d995372ca59fb13045a096221c1 WatchSource:0}: Error finding container a25d745a60e3eab86a40bd49b35ad167f71d7d995372ca59fb13045a096221c1: Status 404 returned error can't find the container with id a25d745a60e3eab86a40bd49b35ad167f71d7d995372ca59fb13045a096221c1
Sep 10 21:36:47.707280 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.707170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" event={"ID":"0f33d12e-b354-4834-9217-f449b01fd12c","Type":"ContainerStarted","Data":"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"}
Sep 10 21:36:47.707280 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.707229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" event={"ID":"0f33d12e-b354-4834-9217-f449b01fd12c","Type":"ContainerStarted","Data":"a25d745a60e3eab86a40bd49b35ad167f71d7d995372ca59fb13045a096221c1"}
Sep 10 21:36:47.707519 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.707291    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:36:47.708663 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.708639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" event={"ID":"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75","Type":"ContainerStarted","Data":"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"}
Sep 10 21:36:47.708784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.708677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" event={"ID":"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75","Type":"ContainerStarted","Data":"ad62564a1afe4cf52ef2155ad7c103c9e5a474051a1867f17ad6857e7ab370ec"}
Sep 10 21:36:47.708784 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.708746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:47.722910 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.722859    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" podStartSLOduration=1.722842266 podStartE2EDuration="1.722842266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:47.721725844 +0000 UTC m=+1625.600434903" watchObservedRunningTime="2026-09-10 21:36:47.722842266 +0000 UTC m=+1625.601551329"
Sep 10 21:36:47.739909 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:47.739869    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" podStartSLOduration=2.170979163 podStartE2EDuration="2.739854958s" totalImagesPullingTime="568.875795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:47.738340397 +0000 UTC m=+1625.617049460" watchObservedRunningTime="2026-09-10 21:36:47.739854958 +0000 UTC m=+1625.618564020"
Sep 10 21:36:53.719714 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:53.719681    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:36:58.716813 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:36:58.716785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:37:16.224737 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.224699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:37:16.225206 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.224955    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" podUID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" containerName="maas-api" containerID="cri-o://1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258" gracePeriod=30
Sep 10 21:37:16.286139 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.286104    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:37:16.286417 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.286366    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" podUID="0f33d12e-b354-4834-9217-f449b01fd12c" containerName="payload-pre-processing" containerID="cri-o://3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588" gracePeriod=30
Sep 10 21:37:16.533585 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.533563    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:37:16.564036 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.564012    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:37:16.676538 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume\") pod \"0f33d12e-b354-4834-9217-f449b01fd12c\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") "
Sep 10 21:37:16.676692 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs\") pod \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") "
Sep 10 21:37:16.676692 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6tb6\" (UniqueName: \"kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6\") pod \"0f33d12e-b354-4834-9217-f449b01fd12c\" (UID: \"0f33d12e-b354-4834-9217-f449b01fd12c\") "
Sep 10 21:37:16.676692 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676627    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk89p\" (UniqueName: \"kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p\") pod \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") "
Sep 10 21:37:16.676692 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676654    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca\") pod \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") "
Sep 10 21:37:16.676878 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") pod \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\" (UID: \"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75\") "
Sep 10 21:37:16.676941 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.676916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f33d12e-b354-4834-9217-f449b01fd12c" (UID: "0f33d12e-b354-4834-9217-f449b01fd12c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:16.677116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.677093    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" (UID: "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:16.677187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.677167    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f33d12e-b354-4834-9217-f449b01fd12c-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.678758 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.678733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p" (OuterVolumeSpecName: "kube-api-access-sk89p") pod "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" (UID: "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75"). InnerVolumeSpecName "kube-api-access-sk89p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:16.678870 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.678758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" (UID: "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:37:16.679130 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.679106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" (UID: "7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:37:16.679130 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.679118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6" (OuterVolumeSpecName: "kube-api-access-q6tb6") pod "0f33d12e-b354-4834-9217-f449b01fd12c" (UID: "0f33d12e-b354-4834-9217-f449b01fd12c"). InnerVolumeSpecName "kube-api-access-q6tb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:16.778317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.778241    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.778317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.778261    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6tb6\" (UniqueName: \"kubernetes.io/projected/0f33d12e-b354-4834-9217-f449b01fd12c-kube-api-access-q6tb6\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.778317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.778289    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk89p\" (UniqueName: \"kubernetes.io/projected/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-kube-api-access-sk89p\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.778317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.778297    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.778317 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.778306    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.858253 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.858223    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f33d12e-b354-4834-9217-f449b01fd12c" containerID="3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588" exitCode=0
Sep 10 21:37:16.858427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.858282    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"
Sep 10 21:37:16.858427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.858321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" event={"ID":"0f33d12e-b354-4834-9217-f449b01fd12c","Type":"ContainerDied","Data":"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"}
Sep 10 21:37:16.858427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.858405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99" event={"ID":"0f33d12e-b354-4834-9217-f449b01fd12c","Type":"ContainerDied","Data":"a25d745a60e3eab86a40bd49b35ad167f71d7d995372ca59fb13045a096221c1"}
Sep 10 21:37:16.858582 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.858435    2580 scope.go:117] "RemoveContainer" containerID="3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"
Sep 10 21:37:16.860106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.860080    2580 generic.go:356] "Generic (PLEG): container finished" podID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" containerID="1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258" exitCode=0
Sep 10 21:37:16.860224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.860113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" event={"ID":"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75","Type":"ContainerDied","Data":"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"}
Sep 10 21:37:16.860224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.860136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"
Sep 10 21:37:16.860224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.860167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5" event={"ID":"7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75","Type":"ContainerDied","Data":"ad62564a1afe4cf52ef2155ad7c103c9e5a474051a1867f17ad6857e7ab370ec"}
Sep 10 21:37:16.869224 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.869203    2580 scope.go:117] "RemoveContainer" containerID="3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"
Sep 10 21:37:16.869541 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:37:16.869526    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588\": container with ID starting with 3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588 not found: ID does not exist" containerID="3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"
Sep 10 21:37:16.869596 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.869550    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588"} err="failed to get container status \"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588\": rpc error: code = NotFound desc = could not find container \"3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588\": container with ID starting with 3ed4a4c6888374953a49d03479bcf060b85de382f94564d6426000105d99a588 not found: ID does not exist"
Sep 10 21:37:16.869596 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.869565    2580 scope.go:117] "RemoveContainer" containerID="1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"
Sep 10 21:37:16.877708 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.877693    2580 scope.go:117] "RemoveContainer" containerID="1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"
Sep 10 21:37:16.877957 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:37:16.877939    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258\": container with ID starting with 1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258 not found: ID does not exist" containerID="1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"
Sep 10 21:37:16.878005 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.877963    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258"} err="failed to get container status \"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258\": rpc error: code = NotFound desc = could not find container \"1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258\": container with ID starting with 1a4cb8b4766f12d04af8a0ef5a04511c6ecc7755a20148475ec198fee9d44258 not found: ID does not exist"
Sep 10 21:37:16.884796 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.884777    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:37:16.888114 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.888096    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-389568-7d869d7d54-t4p99"]
Sep 10 21:37:16.897266 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.897245    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:37:16.900959 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:16.900934    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-389568-5bb4f7578b-j64j5"]
Sep 10 21:37:18.611454 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:18.611424    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f33d12e-b354-4834-9217-f449b01fd12c" path="/var/lib/kubelet/pods/0f33d12e-b354-4834-9217-f449b01fd12c/volumes"
Sep 10 21:37:18.611831 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:18.611817    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" path="/var/lib/kubelet/pods/7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75/volumes"
Sep 10 21:37:41.996097 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:41.996053    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:37:41.996712 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:41.996328    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" podUID="8a805821-fe2c-4121-8616-f02e47669f66" containerName="payload-processing" containerID="cri-o://471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c" gracePeriod=30
Sep 10 21:37:42.247948 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.247892    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:37:42.319938 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.319905    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57mz9\" (UniqueName: \"kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9\") pod \"8a805821-fe2c-4121-8616-f02e47669f66\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") "
Sep 10 21:37:42.320076 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.320001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume\") pod \"8a805821-fe2c-4121-8616-f02e47669f66\" (UID: \"8a805821-fe2c-4121-8616-f02e47669f66\") "
Sep 10 21:37:42.320405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.320381    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a805821-fe2c-4121-8616-f02e47669f66" (UID: "8a805821-fe2c-4121-8616-f02e47669f66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:42.321998 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.321976    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9" (OuterVolumeSpecName: "kube-api-access-57mz9") pod "8a805821-fe2c-4121-8616-f02e47669f66" (UID: "8a805821-fe2c-4121-8616-f02e47669f66"). InnerVolumeSpecName "kube-api-access-57mz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:42.421167 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.421142    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57mz9\" (UniqueName: \"kubernetes.io/projected/8a805821-fe2c-4121-8616-f02e47669f66-kube-api-access-57mz9\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.421167 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.421164    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a805821-fe2c-4121-8616-f02e47669f66-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.990859 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.990832    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a805821-fe2c-4121-8616-f02e47669f66" containerID="471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c" exitCode=0
Sep 10 21:37:42.990989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.990866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" event={"ID":"8a805821-fe2c-4121-8616-f02e47669f66","Type":"ContainerDied","Data":"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"}
Sep 10 21:37:42.990989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.990886    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"
Sep 10 21:37:42.990989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.990906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h" event={"ID":"8a805821-fe2c-4121-8616-f02e47669f66","Type":"ContainerDied","Data":"0d6bd660f8ee8d1842efee59308864078bf43b35c55e691fe50a4a9a739b1fee"}
Sep 10 21:37:42.990989 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:42.990935    2580 scope.go:117] "RemoveContainer" containerID="471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"
Sep 10 21:37:43.003489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:43.003134    2580 scope.go:117] "RemoveContainer" containerID="471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"
Sep 10 21:37:43.003489 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:37:43.003395    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c\": container with ID starting with 471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c not found: ID does not exist" containerID="471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"
Sep 10 21:37:43.003489 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:43.003422    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c"} err="failed to get container status \"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c\": rpc error: code = NotFound desc = could not find container \"471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c\": container with ID starting with 471a3db9907553644472d90bbc50313ec1b73bce687f21e9f480f82d26e72d5c not found: ID does not exist"
Sep 10 21:37:43.011686 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:43.011665    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:37:43.014227 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:43.014168    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c8d758-69b867d8ff-hv89h"]
Sep 10 21:37:44.611156 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:37:44.611125    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a805821-fe2c-4121-8616-f02e47669f66" path="/var/lib/kubelet/pods/8a805821-fe2c-4121-8616-f02e47669f66/volumes"
Sep 10 21:38:58.650917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:58.650876    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:38:58.651480 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:58.651088    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" podUID="8f03dd60-205c-41ba-8509-9e6b986c9f67" containerName="maas-api" containerID="cri-o://eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d" gracePeriod=30
Sep 10 21:38:58.712536 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:58.712503    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:38:58.712766 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:58.712719    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" podUID="0116ad7b-2753-4006-8c6b-9713c5210762" containerName="payload-processing" containerID="cri-o://ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2" gracePeriod=30
Sep 10 21:38:58.997418 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:58.997395    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:38:59.000562 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.000531    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:38:59.101117 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101085    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs\") pod \"8f03dd60-205c-41ba-8509-9e6b986c9f67\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") "
Sep 10 21:38:59.101296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101133    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4rht\" (UniqueName: \"kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht\") pod \"0116ad7b-2753-4006-8c6b-9713c5210762\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") "
Sep 10 21:38:59.101296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101205    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca\") pod \"8f03dd60-205c-41ba-8509-9e6b986c9f67\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") "
Sep 10 21:38:59.101296 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101225    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume\") pod \"0116ad7b-2753-4006-8c6b-9713c5210762\" (UID: \"0116ad7b-2753-4006-8c6b-9713c5210762\") "
Sep 10 21:38:59.101456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") pod \"8f03dd60-205c-41ba-8509-9e6b986c9f67\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") "
Sep 10 21:38:59.101456 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xhnk\" (UniqueName: \"kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk\") pod \"8f03dd60-205c-41ba-8509-9e6b986c9f67\" (UID: \"8f03dd60-205c-41ba-8509-9e6b986c9f67\") "
Sep 10 21:38:59.101635 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101608    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f03dd60-205c-41ba-8509-9e6b986c9f67" (UID: "8f03dd60-205c-41ba-8509-9e6b986c9f67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:59.101704 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.101648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0116ad7b-2753-4006-8c6b-9713c5210762" (UID: "0116ad7b-2753-4006-8c6b-9713c5210762"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:59.103510 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.103471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk" (OuterVolumeSpecName: "kube-api-access-8xhnk") pod "8f03dd60-205c-41ba-8509-9e6b986c9f67" (UID: "8f03dd60-205c-41ba-8509-9e6b986c9f67"). InnerVolumeSpecName "kube-api-access-8xhnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:59.103617 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.103520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f03dd60-205c-41ba-8509-9e6b986c9f67" (UID: "8f03dd60-205c-41ba-8509-9e6b986c9f67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:38:59.103792 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.103772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8f03dd60-205c-41ba-8509-9e6b986c9f67" (UID: "8f03dd60-205c-41ba-8509-9e6b986c9f67"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:38:59.103891 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.103870    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht" (OuterVolumeSpecName: "kube-api-access-v4rht") pod "0116ad7b-2753-4006-8c6b-9713c5210762" (UID: "0116ad7b-2753-4006-8c6b-9713c5210762"). InnerVolumeSpecName "kube-api-access-v4rht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202809    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f03dd60-205c-41ba-8509-9e6b986c9f67-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202834    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0116ad7b-2753-4006-8c6b-9713c5210762-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202846    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202854    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xhnk\" (UniqueName: \"kubernetes.io/projected/8f03dd60-205c-41ba-8509-9e6b986c9f67-kube-api-access-8xhnk\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202862    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f03dd60-205c-41ba-8509-9e6b986c9f67-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.202873 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.202871    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4rht\" (UniqueName: \"kubernetes.io/projected/0116ad7b-2753-4006-8c6b-9713c5210762-kube-api-access-v4rht\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.374644 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.374611    2580 generic.go:356] "Generic (PLEG): container finished" podID="0116ad7b-2753-4006-8c6b-9713c5210762" containerID="ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2" exitCode=0
Sep 10 21:38:59.374833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.374662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"
Sep 10 21:38:59.374833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.374703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" event={"ID":"0116ad7b-2753-4006-8c6b-9713c5210762","Type":"ContainerDied","Data":"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"}
Sep 10 21:38:59.374833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.374794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr" event={"ID":"0116ad7b-2753-4006-8c6b-9713c5210762","Type":"ContainerDied","Data":"fb1b9ef14c5583daefd2e98ea4da1dfb777632483758f959abf68bb2a8eeb727"}
Sep 10 21:38:59.374833 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.374826    2580 scope.go:117] "RemoveContainer" containerID="ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"
Sep 10 21:38:59.376159 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.376122    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f03dd60-205c-41ba-8509-9e6b986c9f67" containerID="eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d" exitCode=0
Sep 10 21:38:59.376222 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.376160    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"
Sep 10 21:38:59.376300 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.376207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" event={"ID":"8f03dd60-205c-41ba-8509-9e6b986c9f67","Type":"ContainerDied","Data":"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"}
Sep 10 21:38:59.376300 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.376265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt" event={"ID":"8f03dd60-205c-41ba-8509-9e6b986c9f67","Type":"ContainerDied","Data":"b95b7bea42b7ccf97364ef37db093315995eb5355c2516b369c564466b8d45a0"}
Sep 10 21:38:59.384917 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.384900    2580 scope.go:117] "RemoveContainer" containerID="ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"
Sep 10 21:38:59.385197 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:38:59.385180    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2\": container with ID starting with ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2 not found: ID does not exist" containerID="ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"
Sep 10 21:38:59.385265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.385205    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2"} err="failed to get container status \"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2\": rpc error: code = NotFound desc = could not find container \"ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2\": container with ID starting with ef35630b4af619bdf74337b658269284be61128eb2caf4b6b7ec23bbb5ebd5b2 not found: ID does not exist"
Sep 10 21:38:59.385265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.385226    2580 scope.go:117] "RemoveContainer" containerID="eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"
Sep 10 21:38:59.393116 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.393097    2580 scope.go:117] "RemoveContainer" containerID="eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"
Sep 10 21:38:59.393415 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:38:59.393395    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d\": container with ID starting with eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d not found: ID does not exist" containerID="eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"
Sep 10 21:38:59.393500 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.393420    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d"} err="failed to get container status \"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d\": rpc error: code = NotFound desc = could not find container \"eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d\": container with ID starting with eed3671c8fe35eed1451aca2f93dcf94dc3d045bf906742c69de7115cd50063d not found: ID does not exist"
Sep 10 21:38:59.398154 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.398134    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:38:59.400435 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.400412    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-36b938-85f8bf575b-g85mt"]
Sep 10 21:38:59.411415 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.411396    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:38:59.414898 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:38:59.414878    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-36b938-588c46657c-hwrmr"]
Sep 10 21:39:00.612189 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:00.612155    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0116ad7b-2753-4006-8c6b-9713c5210762" path="/var/lib/kubelet/pods/0116ad7b-2753-4006-8c6b-9713c5210762/volumes"
Sep 10 21:39:00.612553 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:00.612534    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f03dd60-205c-41ba-8509-9e6b986c9f67" path="/var/lib/kubelet/pods/8f03dd60-205c-41ba-8509-9e6b986c9f67/volumes"
Sep 10 21:39:16.128106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.128068    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:39:16.129063 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.129003    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" podUID="e35abf66-4377-496e-ae0d-2007e88e021d" containerName="maas-api" containerID="cri-o://dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98" gracePeriod=30
Sep 10 21:39:16.197468 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.197204    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:39:16.197639 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.197484    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" podUID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" containerName="payload-pre-processing" containerID="cri-o://9a02ac133056c691262693b40ec806c2f5a97366a97c6fa88b88ca120b8c21ed" gracePeriod=30
Sep 10 21:39:16.436881 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.436858    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:39:16.463416 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.463389    2580 generic.go:356] "Generic (PLEG): container finished" podID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" containerID="9a02ac133056c691262693b40ec806c2f5a97366a97c6fa88b88ca120b8c21ed" exitCode=0
Sep 10 21:39:16.463572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.463475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" event={"ID":"eaefc7b6-48d8-48a7-9c8a-e412d115868e","Type":"ContainerDied","Data":"9a02ac133056c691262693b40ec806c2f5a97366a97c6fa88b88ca120b8c21ed"}
Sep 10 21:39:16.463572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.463539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq" event={"ID":"eaefc7b6-48d8-48a7-9c8a-e412d115868e","Type":"ContainerDied","Data":"af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d"}
Sep 10 21:39:16.463572 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.463562    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af458c6a0aad190ea06accd082504049133c2edcfc117f1fbb5643d0b35d0b8d"
Sep 10 21:39:16.465244 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.465218    2580 generic.go:356] "Generic (PLEG): container finished" podID="e35abf66-4377-496e-ae0d-2007e88e021d" containerID="dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98" exitCode=0
Sep 10 21:39:16.465405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.465266    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"
Sep 10 21:39:16.465405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.465256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" event={"ID":"e35abf66-4377-496e-ae0d-2007e88e021d","Type":"ContainerDied","Data":"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"}
Sep 10 21:39:16.465405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.465345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw" event={"ID":"e35abf66-4377-496e-ae0d-2007e88e021d","Type":"ContainerDied","Data":"119c533b75a21863b9b7d64e7f8f7d15bcd3470c8ad8af8305cd1a892fc6eb98"}
Sep 10 21:39:16.465405 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.465376    2580 scope.go:117] "RemoveContainer" containerID="dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"
Sep 10 21:39:16.469568 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.469551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:39:16.476446 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.476432    2580 scope.go:117] "RemoveContainer" containerID="dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"
Sep 10 21:39:16.476730 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:39:16.476698    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98\": container with ID starting with dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98 not found: ID does not exist" containerID="dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"
Sep 10 21:39:16.476842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.476747    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98"} err="failed to get container status \"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98\": rpc error: code = NotFound desc = could not find container \"dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98\": container with ID starting with dba78f89584aae56b0828ed8475910d8a862c750cd95179a0b38dcf61495df98 not found: ID does not exist"
Sep 10 21:39:16.546943 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.546916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca\") pod \"e35abf66-4377-496e-ae0d-2007e88e021d\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") "
Sep 10 21:39:16.547106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.546963    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs\") pod \"e35abf66-4377-496e-ae0d-2007e88e021d\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") "
Sep 10 21:39:16.547106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqvfk\" (UniqueName: \"kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk\") pod \"e35abf66-4377-496e-ae0d-2007e88e021d\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") "
Sep 10 21:39:16.547106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") pod \"e35abf66-4377-496e-ae0d-2007e88e021d\" (UID: \"e35abf66-4377-496e-ae0d-2007e88e021d\") "
Sep 10 21:39:16.547106 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547073    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dgsv\" (UniqueName: \"kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv\") pod \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") "
Sep 10 21:39:16.547394 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume\") pod \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\" (UID: \"eaefc7b6-48d8-48a7-9c8a-e412d115868e\") "
Sep 10 21:39:16.547394 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e35abf66-4377-496e-ae0d-2007e88e021d" (UID: "e35abf66-4377-496e-ae0d-2007e88e021d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:16.547597 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547576    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35abf66-4377-496e-ae0d-2007e88e021d-openshift-service-ca\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.547672 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.547608    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eaefc7b6-48d8-48a7-9c8a-e412d115868e" (UID: "eaefc7b6-48d8-48a7-9c8a-e412d115868e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:16.549223 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.549199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e35abf66-4377-496e-ae0d-2007e88e021d" (UID: "e35abf66-4377-496e-ae0d-2007e88e021d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:39:16.549223 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.549206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk" (OuterVolumeSpecName: "kube-api-access-xqvfk") pod "e35abf66-4377-496e-ae0d-2007e88e021d" (UID: "e35abf66-4377-496e-ae0d-2007e88e021d"). InnerVolumeSpecName "kube-api-access-xqvfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:16.549392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.549236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e35abf66-4377-496e-ae0d-2007e88e021d" (UID: "e35abf66-4377-496e-ae0d-2007e88e021d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:39:16.549392 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.549253    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv" (OuterVolumeSpecName: "kube-api-access-8dgsv") pod "eaefc7b6-48d8-48a7-9c8a-e412d115868e" (UID: "eaefc7b6-48d8-48a7-9c8a-e412d115868e"). InnerVolumeSpecName "kube-api-access-8dgsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:16.648382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.648326    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-metrics-certs\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.648382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.648348    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqvfk\" (UniqueName: \"kubernetes.io/projected/e35abf66-4377-496e-ae0d-2007e88e021d-kube-api-access-xqvfk\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.648382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.648361    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35abf66-4377-496e-ae0d-2007e88e021d-maas-api-tls\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.648382 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.648374    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dgsv\" (UniqueName: \"kubernetes.io/projected/eaefc7b6-48d8-48a7-9c8a-e412d115868e-kube-api-access-8dgsv\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.648561 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.648387    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaefc7b6-48d8-48a7-9c8a-e412d115868e-plugins-config-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.781427 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.781399    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:39:16.785443 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:16.785422    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1f3879-54b98fd678-lmkkw"]
Sep 10 21:39:17.472410 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:17.472333    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"
Sep 10 21:39:17.486788 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:17.486761    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:39:17.490744 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:17.490722    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1f3879-556bdd6c58-lq2cq"]
Sep 10 21:39:18.611055 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:18.611022    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e35abf66-4377-496e-ae0d-2007e88e021d" path="/var/lib/kubelet/pods/e35abf66-4377-496e-ae0d-2007e88e021d/volumes"
Sep 10 21:39:18.611447 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:18.611429    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" path="/var/lib/kubelet/pods/eaefc7b6-48d8-48a7-9c8a-e412d115868e/volumes"
Sep 10 21:39:42.724652 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:42.724624    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:39:42.727534 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:42.727512    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:39:42.892820 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:39:42.892794    2580 scope.go:117] "RemoveContainer" containerID="9a02ac133056c691262693b40ec806c2f5a97366a97c6fa88b88ca120b8c21ed"
Sep 10 21:41:48.248720 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:48.248687    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:41:48.251175 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:48.248898    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" podUID="d1da87e6-d64f-4cc6-b467-51e6143209df" containerName="manager" containerID="cri-o://1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19" gracePeriod=10
Sep 10 21:41:50.690203 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.690180    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:41:50.749914 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.749835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh4t9\" (UniqueName: \"kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9\") pod \"d1da87e6-d64f-4cc6-b467-51e6143209df\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") "
Sep 10 21:41:50.749914 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.749893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume\") pod \"d1da87e6-d64f-4cc6-b467-51e6143209df\" (UID: \"d1da87e6-d64f-4cc6-b467-51e6143209df\") "
Sep 10 21:41:50.750367 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.750337    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d1da87e6-d64f-4cc6-b467-51e6143209df" (UID: "d1da87e6-d64f-4cc6-b467-51e6143209df"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:50.751843 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.751813    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9" (OuterVolumeSpecName: "kube-api-access-nh4t9") pod "d1da87e6-d64f-4cc6-b467-51e6143209df" (UID: "d1da87e6-d64f-4cc6-b467-51e6143209df"). InnerVolumeSpecName "kube-api-access-nh4t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:41:50.851331 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.851301    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1da87e6-d64f-4cc6-b467-51e6143209df-extensions-socket-volume\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:41:50.851331 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:50.851323    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh4t9\" (UniqueName: \"kubernetes.io/projected/d1da87e6-d64f-4cc6-b467-51e6143209df-kube-api-access-nh4t9\") on node \"ip-10-0-131-178.ec2.internal\" DevicePath \"\""
Sep 10 21:41:51.222253 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.222227    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1da87e6-d64f-4cc6-b467-51e6143209df" containerID="1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19" exitCode=0
Sep 10 21:41:51.222425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.222267    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"
Sep 10 21:41:51.222425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.222304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" event={"ID":"d1da87e6-d64f-4cc6-b467-51e6143209df","Type":"ContainerDied","Data":"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"}
Sep 10 21:41:51.222425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.222353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45" event={"ID":"d1da87e6-d64f-4cc6-b467-51e6143209df","Type":"ContainerDied","Data":"9b21f12e8d8192f1c4aea5841f7495b083e18b74d379d5c6aef13e87da4d67d5"}
Sep 10 21:41:51.222425 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.222375    2580 scope.go:117] "RemoveContainer" containerID="1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"
Sep 10 21:41:51.237515 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.237496    2580 scope.go:117] "RemoveContainer" containerID="1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"
Sep 10 21:41:51.237906 ip-10-0-131-178 kubenswrapper[2580]: E0910 21:41:51.237786    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19\": container with ID starting with 1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19 not found: ID does not exist" containerID="1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"
Sep 10 21:41:51.237906 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.237814    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19"} err="failed to get container status \"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19\": rpc error: code = NotFound desc = could not find container \"1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19\": container with ID starting with 1951021298ed84b53052091a9fa9b048f5df0006ee9f2a906f28bf0425292c19 not found: ID does not exist"
Sep 10 21:41:51.250605 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.250581    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:41:51.254186 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:51.254165    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dmq45"]
Sep 10 21:41:52.612958 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:41:52.612931    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1da87e6-d64f-4cc6-b467-51e6143209df" path="/var/lib/kubelet/pods/d1da87e6-d64f-4cc6-b467-51e6143209df/volumes"
Sep 10 21:42:48.922757 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.922721    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"]
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923063    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" containerName="payload-pre-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923073    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" containerName="payload-pre-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923080    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35abf66-4377-496e-ae0d-2007e88e021d" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923086    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35abf66-4377-496e-ae0d-2007e88e021d" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923096    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f03dd60-205c-41ba-8509-9e6b986c9f67" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923101    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f03dd60-205c-41ba-8509-9e6b986c9f67" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923109    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1da87e6-d64f-4cc6-b467-51e6143209df" containerName="manager"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1da87e6-d64f-4cc6-b467-51e6143209df" containerName="manager"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923121    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0116ad7b-2753-4006-8c6b-9713c5210762" containerName="payload-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923126    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0116ad7b-2753-4006-8c6b-9713c5210762" containerName="payload-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923133    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923138    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" containerName="maas-api"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923146    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f33d12e-b354-4834-9217-f449b01fd12c" containerName="payload-pre-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923151    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f33d12e-b354-4834-9217-f449b01fd12c" containerName="payload-pre-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923162    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a805821-fe2c-4121-8616-f02e47669f66" containerName="payload-processing"
Sep 10 21:42:48.923161 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923168    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a805821-fe2c-4121-8616-f02e47669f66" containerName="payload-processing"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923234    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cb719f8-3cd4-4b8c-a5ac-7417dd8d1e75" containerName="maas-api"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923241    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaefc7b6-48d8-48a7-9c8a-e412d115868e" containerName="payload-pre-processing"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923247    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1da87e6-d64f-4cc6-b467-51e6143209df" containerName="manager"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923252    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f03dd60-205c-41ba-8509-9e6b986c9f67" containerName="maas-api"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923258    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0116ad7b-2753-4006-8c6b-9713c5210762" containerName="payload-processing"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923263    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f33d12e-b354-4834-9217-f449b01fd12c" containerName="payload-pre-processing"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923282    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a805821-fe2c-4121-8616-f02e47669f66" containerName="payload-processing"
Sep 10 21:42:48.923693 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.923287    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35abf66-4377-496e-ae0d-2007e88e021d" containerName="maas-api"
Sep 10 21:42:48.926176 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.926157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:48.929253 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.929222    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x27fp\""
Sep 10 21:42:48.934649 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.934625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:48.934749 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.934679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9562\" (UniqueName: \"kubernetes.io/projected/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-kube-api-access-z9562\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:48.940720 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:48.940693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"]
Sep 10 21:42:49.036089 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.036063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:49.036233 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.036117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9562\" (UniqueName: \"kubernetes.io/projected/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-kube-api-access-z9562\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:49.036450 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.036432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:49.046575 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.046546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9562\" (UniqueName: \"kubernetes.io/projected/a9ce5cfc-b433-4fbd-a5f2-561871bf6361-kube-api-access-z9562\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ts28l\" (UID: \"a9ce5cfc-b433-4fbd-a5f2-561871bf6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:49.236882 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.236802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:49.513926 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:49.513901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"]
Sep 10 21:42:49.516561 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:42:49.516536    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ce5cfc_b433_4fbd_a5f2_561871bf6361.slice/crio-13d4378e2271c4516845e3e044eb2d1fa21e128ddbf30d1199e503e3b5c674bd WatchSource:0}: Error finding container 13d4378e2271c4516845e3e044eb2d1fa21e128ddbf30d1199e503e3b5c674bd: Status 404 returned error can't find the container with id 13d4378e2271c4516845e3e044eb2d1fa21e128ddbf30d1199e503e3b5c674bd
Sep 10 21:42:50.501643 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:50.501604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l" event={"ID":"a9ce5cfc-b433-4fbd-a5f2-561871bf6361","Type":"ContainerStarted","Data":"6b2e36adef50b919ee9c240a00f7ea10e16a6d92d175feaaea687b7fbcc11694"}
Sep 10 21:42:50.502031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:50.501664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l" event={"ID":"a9ce5cfc-b433-4fbd-a5f2-561871bf6361","Type":"ContainerStarted","Data":"13d4378e2271c4516845e3e044eb2d1fa21e128ddbf30d1199e503e3b5c674bd"}
Sep 10 21:42:50.502031 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:50.501831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:42:50.522773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:42:50.522731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l" podStartSLOduration=2.5227156600000002 podStartE2EDuration="2.52271566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:42:50.520095668 +0000 UTC m=+1988.398804729" watchObservedRunningTime="2026-09-10 21:42:50.52271566 +0000 UTC m=+1988.401424724"
Sep 10 21:43:01.508399 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:43:01.508365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ts28l"
Sep 10 21:44:36.357450 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.357418    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"]
Sep 10 21:44:36.360709 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.360690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.363514 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.363480    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:44:36.363514 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.363480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:44:36.363731 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.363520    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:44:36.363731 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.363481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:44:36.363731 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.363481    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:44:36.370698 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.370660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"]
Sep 10 21:44:36.450095 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.450072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5b4\" (UniqueName: \"kubernetes.io/projected/d4a27035-20ef-4b35-9b90-406c31f55031-kube-api-access-vb5b4\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.450221 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.450136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-metrics-certs\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.450221 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.450159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-maas-api-tls\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.450221 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.450173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a27035-20ef-4b35-9b90-406c31f55031-openshift-service-ca\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.551642 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.551612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-metrics-certs\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.551773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.551659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-maas-api-tls\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.551773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.551685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a27035-20ef-4b35-9b90-406c31f55031-openshift-service-ca\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.551773 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.551762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5b4\" (UniqueName: \"kubernetes.io/projected/d4a27035-20ef-4b35-9b90-406c31f55031-kube-api-access-vb5b4\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.552467 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.552448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a27035-20ef-4b35-9b90-406c31f55031-openshift-service-ca\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.554695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.554675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-metrics-certs\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.554795 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.554748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4a27035-20ef-4b35-9b90-406c31f55031-maas-api-tls\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.560454 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.560435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5b4\" (UniqueName: \"kubernetes.io/projected/d4a27035-20ef-4b35-9b90-406c31f55031-kube-api-access-vb5b4\") pod \"maas-api-84bdd77b86-q2rjl\" (UID: \"d4a27035-20ef-4b35-9b90-406c31f55031\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.672339 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.672310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:36.942575 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.942551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"]
Sep 10 21:44:36.944665 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:44:36.944631    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a27035_20ef_4b35_9b90_406c31f55031.slice/crio-90cd021ef2ce88127fba32defae143a22960f1a4d3b1cce245def77c726ca920 WatchSource:0}: Error finding container 90cd021ef2ce88127fba32defae143a22960f1a4d3b1cce245def77c726ca920: Status 404 returned error can't find the container with id 90cd021ef2ce88127fba32defae143a22960f1a4d3b1cce245def77c726ca920
Sep 10 21:44:36.946040 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:36.946019    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:44:37.014264 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:37.014231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl" event={"ID":"d4a27035-20ef-4b35-9b90-406c31f55031","Type":"ContainerStarted","Data":"90cd021ef2ce88127fba32defae143a22960f1a4d3b1cce245def77c726ca920"}
Sep 10 21:44:38.020639 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:38.020602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl" event={"ID":"d4a27035-20ef-4b35-9b90-406c31f55031","Type":"ContainerStarted","Data":"b5c885b988a0c38d5592ee166650357ae5bbf023718afa810f428412f4d99ff2"}
Sep 10 21:44:38.021052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:38.020727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:44:38.038996 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:38.038952    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl" podStartSLOduration=1.671905872 podStartE2EDuration="2.038939242s" totalImagesPullingTime="367.03337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:44:38.037036217 +0000 UTC m=+2095.915745291" watchObservedRunningTime="2026-09-10 21:44:38.038939242 +0000 UTC m=+2095.917648305"
Sep 10 21:44:42.754460 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:42.754431    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:44:42.758806 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:42.758782    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dshvv_1de57c84-3f66-4c7e-8815-e8beb0ac1fb4/ovn-acl-logging/0.log"
Sep 10 21:44:44.031052 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:44:44.031023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-q2rjl"
Sep 10 21:45:03.052018 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:03.051990    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-drxqt_eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6/manager/0.log"
Sep 10 21:45:03.166265 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:03.166234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4wj6q_fa853146-004c-468b-9cb8-dc5a8e0cc396/manager/0.log"
Sep 10 21:45:03.396516 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:03.396495    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wslc9_a159db2f-b1f3-431b-a52f-623757613c20/server/0.log"
Sep 10 21:45:03.621970 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:03.621943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj_f734446d-8895-4042-b0c4-0228541dc04b/copy-manifests/0.log"
Sep 10 21:45:03.631311 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:03.631287    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj_f734446d-8895-4042-b0c4-0228541dc04b/manager/0.log"
Sep 10 21:45:04.205182 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:04.205153    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-84bdd77b86-q2rjl_d4a27035-20ef-4b35-9b90-406c31f55031/maas-api/0.log"
Sep 10 21:45:05.471797 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:05.471771    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-84r26_707e1800-cdf3-43d7-95c5-d89f92cf3cde/manager/0.log"
Sep 10 21:45:05.804743 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:05.804672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ts28l_a9ce5cfc-b433-4fbd-a5f2-561871bf6361/manager/0.log"
Sep 10 21:45:07.508939 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:07.508912    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzqf_a384ab32-fd70-4c81-b1a4-ecf3e23b9456/main/0.log"
Sep 10 21:45:07.729589 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:07.729561    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-hf6wf_8d93bf61-7e78-4d99-943c-bdbe7b3c5565/main/0.log"
Sep 10 21:45:08.078151 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:08.078117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nxjd7_5a725897-272e-4ba1-a4fa-24776df980c6/main/0.log"
Sep 10 21:45:08.182187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:08.182159    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2gw2s_c4a11080-869c-4136-997e-8c948e510796/main/0.log"
Sep 10 21:45:16.319703 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:16.319672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kjzvc_0dc55b83-b0f4-4f7f-a39e-2aae6cb8c668/global-pull-secret-syncer/0.log"
Sep 10 21:45:16.564997 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:16.564970    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xl8x9_e94b1464-e117-4798-ae62-c4e8dbe02f75/konnectivity-agent/0.log"
Sep 10 21:45:16.605386 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:16.605322    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-178.ec2.internal_52c9ae203a047aa285585b14707a01ad/haproxy/0.log"
Sep 10 21:45:21.454825 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:21.454795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-84r26_707e1800-cdf3-43d7-95c5-d89f92cf3cde/manager/0.log"
Sep 10 21:45:21.567764 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:21.567732    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ts28l_a9ce5cfc-b433-4fbd-a5f2-561871bf6361/manager/0.log"
Sep 10 21:45:23.319389 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.319365    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lfcm7_adfd4b2e-4a1f-4f5f-8002-d5afccc21606/kube-state-metrics/0.log"
Sep 10 21:45:23.345117 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.345097    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lfcm7_adfd4b2e-4a1f-4f5f-8002-d5afccc21606/kube-rbac-proxy-main/0.log"
Sep 10 21:45:23.368187 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.368165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lfcm7_adfd4b2e-4a1f-4f5f-8002-d5afccc21606/kube-rbac-proxy-self/0.log"
Sep 10 21:45:23.396818 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.396795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b98cbb57b-lp5fx_c3c5d3a5-eea9-4013-a696-6026b445195c/metrics-server/0.log"
Sep 10 21:45:23.698032 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.698004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xpgt8_b8482532-6043-4222-a5f3-491e8c032883/node-exporter/0.log"
Sep 10 21:45:23.728005 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.727982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xpgt8_b8482532-6043-4222-a5f3-491e8c032883/kube-rbac-proxy/0.log"
Sep 10 21:45:23.752485 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.752460    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xpgt8_b8482532-6043-4222-a5f3-491e8c032883/init-textfile/0.log"
Sep 10 21:45:23.885049 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.885028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/prometheus/0.log"
Sep 10 21:45:23.911228 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.911209    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/config-reloader/0.log"
Sep 10 21:45:23.935095 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.935078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/thanos-sidecar/0.log"
Sep 10 21:45:23.957373 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.957349    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/kube-rbac-proxy-web/0.log"
Sep 10 21:45:23.985301 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:23.985263    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/kube-rbac-proxy/0.log"
Sep 10 21:45:24.004226 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.004200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/kube-rbac-proxy-thanos/0.log"
Sep 10 21:45:24.029761 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.029737    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9523f6ee-e4a1-48d2-8acc-0b5073e2b38c/init-config-reloader/0.log"
Sep 10 21:45:24.117822 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.117799    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-9r5sr_79a4b2be-aaef-436d-b171-02a3ce4b8e67/prometheus-operator-admission-webhook/0.log"
Sep 10 21:45:24.242884 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.242810    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/thanos-query/0.log"
Sep 10 21:45:24.269494 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.269472    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/kube-rbac-proxy-web/0.log"
Sep 10 21:45:24.295842 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.295819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/kube-rbac-proxy/0.log"
Sep 10 21:45:24.317288 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.317238    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/prom-label-proxy/0.log"
Sep 10 21:45:24.339783 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.339764    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/kube-rbac-proxy-rules/0.log"
Sep 10 21:45:24.363259 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:24.363237    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5d5996d8-69h94_89ef9e24-aada-4e70-be3c-3d6a44faf1d6/kube-rbac-proxy-metrics/0.log"
Sep 10 21:45:26.514949 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.514913    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"]
Sep 10 21:45:26.518695 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.518674    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.520559 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.520539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"openshift-service-ca.crt\""
Sep 10 21:45:26.520683 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.520547    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vms7j\"/\"default-dockercfg-jtgj9\""
Sep 10 21:45:26.521153 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.521139    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"kube-root-ca.crt\""
Sep 10 21:45:26.525526 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.525499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"]
Sep 10 21:45:26.586819 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.586785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z27lh\" (UniqueName: \"kubernetes.io/projected/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-kube-api-access-z27lh\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.586980 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.586837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-lib-modules\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.586980 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.586892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-sys\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.586980 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.586959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-podres\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.587102 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.586995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-proc\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.687947 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.687915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z27lh\" (UniqueName: \"kubernetes.io/projected/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-kube-api-access-z27lh\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.687952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-lib-modules\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.687973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-sys\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-podres\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-proc\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-lib-modules\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688122 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-sys\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688421 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-podres\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.688421 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.688164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-proc\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.696860 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.696837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z27lh\" (UniqueName: \"kubernetes.io/projected/a8e5d307-048e-4e5b-8b68-b58d9a70d4f9-kube-api-access-z27lh\") pod \"perf-node-gather-daemonset-p4cn6\" (UID: \"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:26.829776 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:26.829698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:27.092479 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.092410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-wxjs5_b5cd1942-52c2-4dbd-8a8b-e022c1d289a8/download-server/0.log"
Sep 10 21:45:27.101484 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.101359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"]
Sep 10 21:45:27.104227 ip-10-0-131-178 kubenswrapper[2580]: W0910 21:45:27.104194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda8e5d307_048e_4e5b_8b68_b58d9a70d4f9.slice/crio-58d75aa310937fef25624309068ddf4e90b012c14c8f4be01544125073af7369 WatchSource:0}: Error finding container 58d75aa310937fef25624309068ddf4e90b012c14c8f4be01544125073af7369: Status 404 returned error can't find the container with id 58d75aa310937fef25624309068ddf4e90b012c14c8f4be01544125073af7369
Sep 10 21:45:27.257524 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.257490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6" event={"ID":"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9","Type":"ContainerStarted","Data":"60114c2d675b5ceeaf6b32a736a805736864edd70ab64daaed2fd651c59ee1f0"}
Sep 10 21:45:27.257683 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.257543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6" event={"ID":"a8e5d307-048e-4e5b-8b68-b58d9a70d4f9","Type":"ContainerStarted","Data":"58d75aa310937fef25624309068ddf4e90b012c14c8f4be01544125073af7369"}
Sep 10 21:45:27.257683 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.257599    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:27.272261 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.272214    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6" podStartSLOduration=1.272190671 podStartE2EDuration="1.272190671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:27.27157565 +0000 UTC m=+2145.150284711" watchObservedRunningTime="2026-09-10 21:45:27.272190671 +0000 UTC m=+2145.150899735"
Sep 10 21:45:27.797876 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:27.797849    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-8tsbz_2621728c-025f-489c-a811-97f005237bc1/volume-data-source-validator/0.log"
Sep 10 21:45:28.884584 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:28.884553    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b8h6t_550f50e1-7b22-4f80-809a-43044ae513ac/dns/0.log"
Sep 10 21:45:28.909971 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:28.909947    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b8h6t_550f50e1-7b22-4f80-809a-43044ae513ac/kube-rbac-proxy/0.log"
Sep 10 21:45:28.994414 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:28.994380    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5zqd4_a4040f5e-8b82-4b38-ab74-7e50328e2ab0/dns-node-resolver/0.log"
Sep 10 21:45:29.634192 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:29.634164    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fg6nr_a8efe91e-dab3-4f79-882b-3e8e13c07992/node-ca/0.log"
Sep 10 21:45:32.126998 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:32.126965    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpg2v_714039b9-eb08-4aa6-90c9-260cfb76b60c/serve-healthcheck-canary/0.log"
Sep 10 21:45:32.749009 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:32.748976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2pbk_87e9d8a2-47df-4f6a-b538-4c19d54dd1db/kube-rbac-proxy/0.log"
Sep 10 21:45:32.771004 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:32.770978    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2pbk_87e9d8a2-47df-4f6a-b538-4c19d54dd1db/exporter/0.log"
Sep 10 21:45:32.793449 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:32.793422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2pbk_87e9d8a2-47df-4f6a-b538-4c19d54dd1db/extractor/0.log"
Sep 10 21:45:33.271810 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:33.271785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-p4cn6"
Sep 10 21:45:34.950101 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:34.950069    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-drxqt_eda92e62-8e3d-44ed-b5bf-031c7d2aa0c6/manager/0.log"
Sep 10 21:45:35.120948 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:35.120889    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4wj6q_fa853146-004c-468b-9cb8-dc5a8e0cc396/manager/0.log"
Sep 10 21:45:35.211815 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:35.211751    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wslc9_a159db2f-b1f3-431b-a52f-623757613c20/server/0.log"
Sep 10 21:45:35.366929 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:35.366883    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj_f734446d-8895-4042-b0c4-0228541dc04b/manager/0.log"
Sep 10 21:45:35.387810 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:35.387787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-54kpj_f734446d-8895-4042-b0c4-0228541dc04b/copy-manifests/0.log"
Sep 10 21:45:36.907200 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:36.907170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66f6f49f76-66srw_1b1748df-2811-4a3e-a5c7-6e2ba57f198e/manager/0.log"
Sep 10 21:45:36.959539 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:36.959506    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-np8xn_071f660f-e1f3-4219-aee5-28260d566d9e/openshift-lws-operator/0.log"
Sep 10 21:45:42.088282 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:42.088240    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-l22hq_fcc9febc-b7d0-4695-98a4-aa0d4329d32a/migrator/0.log"
Sep 10 21:45:42.111863 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:42.111838    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-l22hq_fcc9febc-b7d0-4695-98a4-aa0d4329d32a/graceful-termination/0.log"
Sep 10 21:45:43.639605 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.639575    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/kube-multus-additional-cni-plugins/0.log"
Sep 10 21:45:43.663840 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.663814    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/egress-router-binary-copy/0.log"
Sep 10 21:45:43.685516 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.685454    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/cni-plugins/0.log"
Sep 10 21:45:43.707979 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.707958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/bond-cni-plugin/0.log"
Sep 10 21:45:43.729732 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.729713    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/routeoverride-cni/0.log"
Sep 10 21:45:43.752073 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.752049    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/whereabouts-cni-bincopy/0.log"
Sep 10 21:45:43.775944 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.775924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zwbc_76912b7e-9cdd-4879-a340-e66ca7c0c79c/whereabouts-cni/0.log"
Sep 10 21:45:43.983611 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:43.983539    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cbngr_fcfa4e0b-f081-4378-898f-8c5e737ed0c7/kube-multus/0.log"
Sep 10 21:45:44.121184 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:44.121136    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p2vbb_ef6a4e86-206d-4dba-86cd-b033c338d8b3/network-metrics-daemon/0.log"
Sep 10 21:45:44.140282 ip-10-0-131-178 kubenswrapper[2580]: I0910 21:45:44.140258    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p2vbb_ef6a4e86-206d-4dba-86cd-b033c338d8b3/kube-rbac-proxy/0.log"