Sep 10 23:36:49.625766 ip-10-0-134-115 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 23:36:49.999652 ip-10-0-134-115 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:36:49.999652 ip-10-0-134-115 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 23:36:49.999652 ip-10-0-134-115 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:36:49.999652 ip-10-0-134-115 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 23:36:49.999652 ip-10-0-134-115 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:36:50.001203 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.001117    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 23:36:50.003831 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.003802    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.004346 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004329    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.004468 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004458    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.004573 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004564    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.004630 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004624    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.004679 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004673    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.004734 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004710    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.004734 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004729    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.004734 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004735    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.004848 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004740    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.004848 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004745    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.004848 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004750    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.004848 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004808    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.004944 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004927    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.004944 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004932    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.004944 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004936    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.004944 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004942    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004948    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004951    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004954    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004957    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004960    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004963    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004965    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004969    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004971    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004974    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004977    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004979    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004982    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004985    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004987    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004989    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004992    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.004996    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.005042 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005001    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005004    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005007    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005013    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005015    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005018    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005020    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005022    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005025    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005027    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005032    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005035    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005038    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005040    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005043    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005046    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005049    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005052    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005054    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005057    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005059    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.005540 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005063    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005066    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005068    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005070    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005073    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005076    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005078    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005081    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005083    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005085    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005088    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005090    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005093    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005096    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005098    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005101    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005103    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005106    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005108    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.006130 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005111    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005113    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005116    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005118    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005120    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005122    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005125    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005128    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005132    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005134    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005137    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005139    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005142    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005144    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005147    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005149    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005151    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005154    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005156    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005159    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.006587 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005161    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005164    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005168    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005170    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005885    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005893    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005897    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005900    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005904    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005906    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005909    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005913    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005917    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005920    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005923    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005926    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005928    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005931    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005934    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.007053 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005937    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005939    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005943    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005945    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005948    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005951    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005953    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005955    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005958    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005960    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005963    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005966    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005968    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005971    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005975    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005979    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005982    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005985    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005987    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.007514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005990    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005993    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005996    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.005999    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006002    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006005    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006008    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006011    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006013    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006016    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006018    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006021    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006023    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006026    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006028    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006030    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006033    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006035    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006038    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.007980 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006041    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006044    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006047    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006050    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006053    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006056    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006059    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006062    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006065    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006067    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006070    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006073    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006075    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006078    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006080    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006083    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006085    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006088    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006091    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006093    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.008447 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006096    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006098    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006101    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006104    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006107    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006109    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006112    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006114    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006116    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006119    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006122    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006125    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006127    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006129    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006132    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006134    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006137    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006140    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006142    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006145    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.008956 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006147    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006150    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006152    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006155    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006157    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006159    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006555    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006560    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006563    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006565    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006568    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006571    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006573    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006576    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006578    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006581    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006583    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006588    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006592    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.009451 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006594    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006597    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006600    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006603    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006606    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006608    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006610    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006613    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006615    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006618    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006621    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006623    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006626    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006628    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006631    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006634    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006636    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006639    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006641    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006644    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.009912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006646    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006649    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006651    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006654    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006657    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006660    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006666    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006670    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006673    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006676    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006678    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006682    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006685    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006688    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006690    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006693    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006696    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006698    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006701    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006703    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.010418 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006706    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006708    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006711    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006714    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006716    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006719    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006721    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006724    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006726    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006729    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006731    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006734    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006738    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006740    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006743    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006748    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006751    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006753    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006755    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.010919 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006758    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006761    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006764    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006766    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006768    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006771    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006773    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006776    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006778    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006780    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006784    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006786    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006789    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006791    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006793    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006796    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006799    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006801    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006803    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006806    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.011366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006808    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006810    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006813    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006815    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006818    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006820    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.006822    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006900    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006939    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006946    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006951    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006954    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006957    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006961    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006965    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006968    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006971    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006975    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006978    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006981    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006983    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006986    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006989    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 10 23:36:50.011892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006992    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006994    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.006998    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007001    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007004    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007007    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007010    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007014    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007016    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007019    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007023    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007025    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007028    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007031    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007034    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007037    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007041    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007045    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007048    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007050    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007053    2575 flags.go:64] FLAG: --enable-server="true"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007056    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007061    2575 flags.go:64] FLAG: --event-burst="100"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007064    2575 flags.go:64] FLAG: --event-qps="50"
Sep 10 23:36:50.012416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007067    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007070    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007073    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007076    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007079    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007082    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007085    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007087    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007090    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007094    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007097    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007099    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007102    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007105    2575 flags.go:64] FLAG: --feature-gates=""
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007108    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007111    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007114    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007117    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007120    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007123    2575 flags.go:64] FLAG: --help="false"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007125    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007129    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007131    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 23:36:50.013003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007134    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007137    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007141    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007144    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007146    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007149    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007152    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007154    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007157    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007160    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007163    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007165    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007168    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007171    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007173    2575 flags.go:64] FLAG: --lock-file=""
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007176    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007179    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007181    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007186    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007189    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007193    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007195    2575 flags.go:64] FLAG: --logging-format="text"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007198    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007202    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 23:36:50.013897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007205    2575 flags.go:64] FLAG: --manifest-url=""
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007207    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007212    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007214    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007218    2575 flags.go:64] FLAG: --max-pods="110"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007221    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007224    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007227    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007229    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007232    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007235    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007238    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007246    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007249    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007252    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007255    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007258    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007263    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007266    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007269    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007271    2575 flags.go:64] FLAG: --port="10250"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007275    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007277    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c7dbbb8b55948ec1"
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007280    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 10 23:36:50.014908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007283    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007286    2575 flags.go:64] FLAG: --register-node="true"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007288    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007296    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007300    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007302    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007307    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007310    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007313    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007316    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007319    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007321    2575 flags.go:64] FLAG: --runonce="false"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007324    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007327    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007329    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007332    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007335    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007338    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007341    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007343    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007346    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007349    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007351    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007354    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007357    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 23:36:50.015742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007359    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007362    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007367    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007369    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007372    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007376    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007378    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007381    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007383    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007386    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007388    2575 flags.go:64] FLAG: --v="2"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007392    2575 flags.go:64] FLAG: --version="false"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007397    2575 flags.go:64] FLAG: --vmodule=""
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007402    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007405    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007530    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007534    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007537    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007540    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007542    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007546    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007550    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.016519 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007552    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007555    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007558    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007560    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007563    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007565    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007568    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007571    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007573    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007576    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007578    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007580    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007583    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007585    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007588    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007590    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007593    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007596    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007598    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007600    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.017480 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007603    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007605    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007608    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007612    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007615    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007617    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007620    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007623    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007625    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007628    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007631    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007633    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007636    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007638    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007641    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007645    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007648    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007650    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007652    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007655    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.018120 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007657    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007660    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007663    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007665    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007669    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007672    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007674    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007677    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007679    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007682    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007684    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007687    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007689    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007691    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007694    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007698    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007702    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007704    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007707    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.018927 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007709    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007712    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007714    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007717    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007719    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007722    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007724    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007727    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007730    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007733    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007735    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007738    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007740    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007742    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007745    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007748    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007750    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007753    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007755    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007757    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.019657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007760    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007762    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007766    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007770    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007773    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007775    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007778    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007781    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007785    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007787    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007791    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007794    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.007797    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.007802    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.015428    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 23:36:50.020311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.015448    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.016250    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.016264    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016349    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016357    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016362    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016367    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016372    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016376    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016382    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016386    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016390    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016395    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016399    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016403    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016407    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016412    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016415    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016419    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.020717 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016424    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016430    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016436    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016440    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016444    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016448    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016452    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016456    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016462    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016468    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016472    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016476    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016481    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016485    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016506    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016511    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016515    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016518    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016522    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016526    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.021271 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016530    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016533    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016537    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016541    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016545    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016549    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016553    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016564    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016569    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016573    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016577    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016581    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016585    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016589    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016593    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016597    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016601    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016606    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016610    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.021836 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016614    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016618    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016624    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016628    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016632    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016636    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016640    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016645    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016649    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016653    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016657    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016661    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016665    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016669    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016674    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016677    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016681    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016685    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016689    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.022320 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016697    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016702    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016706    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016711    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016715    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016719    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016731    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016735    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016739    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016743    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016748    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016752    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016756    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016760    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016764    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016769    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016774    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016778    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016782    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016786    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.022788 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016791    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016795    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016801    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016806    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016811    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.016819    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.016997    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017006    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017011    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017015    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017020    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017025    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017031    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017035    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:36:50.023657 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017039    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017043    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017047    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017051    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017055    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017059    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017063    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017067    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017071    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017075    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017079    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017083    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017087    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017092    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017095    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017100    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017104    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017108    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017112    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017116    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:36:50.024219 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017120    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017124    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017128    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017132    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017136    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017140    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017143    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017147    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017151    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017155    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017159    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017163    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017167    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017171    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017175    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017179    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017183    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017186    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017190    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:36:50.025050 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017196    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017202    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017206    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017212    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017217    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017221    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017225    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017229    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017233    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017244    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017248    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017253    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017257    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017261    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017264    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017269    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017273    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017277    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017281    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017286    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:36:50.025670 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017290    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017294    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017298    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017302    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017306    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017310    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017314    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017317    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017321    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017325    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017329    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017334    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017338    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017342    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017346    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017350    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017354    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017358    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017362    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017366    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:36:50.026172 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017370    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017374    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017379    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017384    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017388    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017392    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017396    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017400    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017404    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017408    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017412    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:50.017416    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.017423    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.017594    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 23:36:50.026671 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.020017    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 23:36:50.027068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.020894    2575 server.go:1019] "Starting client certificate rotation"
Sep 10 23:36:50.027068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.020994    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 23:36:50.027068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.021032    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 23:36:50.027068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.023458    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 23:36:50.027068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.025317    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 23:36:50.035101 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.035079    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 10 23:36:50.037867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.037850    2575 log.go:25] "Validated CRI v1 image API"
Sep 10 23:36:50.039275 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.039257    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 23:36:50.041788 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.041766    2575 fs.go:135] Filesystem UUIDs: map[076a13a7-8ba8-49e9-803f-8292038c7c33:/dev/nvme0n1p4 0d914897-da2c-42d2-b0bc-1867481fc9aa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 10 23:36:50.041873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.041788    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 23:36:50.047377 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.047261    2575 manager.go:216] Machine: {Timestamp:2026-09-10 23:36:50.045449501 +0000 UTC m=+0.311446381 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200602 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cf71b77dc864a5f943c710b19aa69 SystemUUID:ec2cf71b-77dc-864a-5f94-3c710b19aa69 BootID:614491e5-0c45-4df9-8811-ed3392c903c1 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:eb:ad:ec:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:eb:ad:ec:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:c4:47:4c:e8:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 23:36:50.047377 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.047369    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 23:36:50.047563 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.047481    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 23:36:50.049736 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.049720    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 23:36:50.050250 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.050227    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 23:36:50.050388 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.050254    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 23:36:50.051061 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.051051    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 23:36:50.051099 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.051063    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 23:36:50.051099 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.051070    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 23:36:50.051590 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.051580    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 23:36:50.052235 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.052226    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 23:36:50.055687 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055675    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 23:36:50.055813 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055804    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 23:36:50.055844 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055834    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 23:36:50.055870 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055844    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 23:36:50.055870 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055857    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 23:36:50.055870 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055867    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 10 23:36:50.055958 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.055876    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 23:36:50.056249 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.056238    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 23:36:50.056290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.056260    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 23:36:50.058613 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.058597    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 23:36:50.060282 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.060269    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 23:36:50.060320 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.060305    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 23:36:50.061674 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061659    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 23:36:50.061674 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061677    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061683    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061688    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061701    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061706    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061719    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061723    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061729    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061734    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061748    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 23:36:50.061759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.061753    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 23:36:50.062442 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.062433    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 23:36:50.062473 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.062444    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 23:36:50.065624 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.065604    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 23:36:50.065719 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.065620    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 23:36:50.065719 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.065689    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 23:36:50.066185 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.066172    2575 server.go:1268] "Started kubelet"
Sep 10 23:36:50.066288 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.066258    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 23:36:50.066359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.066315    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 23:36:50.066394 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.066383    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 23:36:50.067511 ip-10-0-134-115 systemd[1]: Started Kubernetes Kubelet.
Sep 10 23:36:50.069995 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.069949    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 23:36:50.071135 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.070957    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 23:36:50.071397 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.071373    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 23:36:50.074008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.073989    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qb8cw"
Sep 10 23:36:50.075480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.075466    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 23:36:50.075480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.075474    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 23:36:50.075607 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.075561    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.076011 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.075988    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 23:36:50.076085 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.075992    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 23:36:50.076085 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076035    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 23:36:50.076193 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076171    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 23:36:50.076193 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076177    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 23:36:50.076425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076408    2575 factory.go:55] Registering systemd factory
Sep 10 23:36:50.076425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076429    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 10 23:36:50.076757 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.075692    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-115.ec2.internal.18d419e1ea898486  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-115.ec2.internal,UID:ip-10-0-134-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-115.ec2.internal,},FirstTimestamp:2026-09-10 23:36:50.066146438 +0000 UTC m=+0.332143299,LastTimestamp:2026-09-10 23:36:50.066146438 +0000 UTC m=+0.332143299,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-115.ec2.internal,}"
Sep 10 23:36:50.076859 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076847    2575 factory.go:153] Registering CRI-O factory
Sep 10 23:36:50.076941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076861    2575 factory.go:223] Registration of the crio container factory successfully
Sep 10 23:36:50.076941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076903    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 23:36:50.076941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076919    2575 factory.go:103] Registering Raw factory
Sep 10 23:36:50.076941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.076931    2575 manager.go:1196] Started watching for new ooms in manager
Sep 10 23:36:50.077330 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.077317    2575 manager.go:349] Starting recovery of all containers
Sep 10 23:36:50.082019 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.081994    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qb8cw"
Sep 10 23:36:50.082952 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.082924    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 23:36:50.083076 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.083052    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 23:36:50.088133 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.088067    2575 manager.go:354] Recovery completed
Sep 10 23:36:50.090039 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.090022    2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 10 23:36:50.092776 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.092764    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.094548 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.094532    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.094616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.094563    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.094616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.094575    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.095113 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095101    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 23:36:50.095113 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095113    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 23:36:50.095221 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095106    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.095221 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095127    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 23:36:50.095799 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095786    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.095871 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095810    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.095871 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.095819    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.096075 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.096062    2575 policy_none.go:49] "None policy: Start"
Sep 10 23:36:50.096130 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.096081    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 23:36:50.096130 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.096094    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 23:36:50.097006 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.096717    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-115.ec2.internal.18d419e1ec3ada86  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-115.ec2.internal,UID:ip-10-0-134-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-115.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-115.ec2.internal,},FirstTimestamp:2026-09-10 23:36:50.094545542 +0000 UTC m=+0.360542403,LastTimestamp:2026-09-10 23:36:50.094545542 +0000 UTC m=+0.360542403,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-115.ec2.internal,}"
Sep 10 23:36:50.097417 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.097393    2575 policy_none.go:47] "Start"
Sep 10 23:36:50.149033 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149017    2575 manager.go:337] "Starting Device Plugin manager"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.149060    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149070    2575 server.go:87] "Starting device plugin registration server"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149392    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149407    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149516    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149586    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 23:36:50.149956 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.149593    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 23:36:50.150276 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.150132    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 23:36:50.150276 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.150182    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.219888 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.219864    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 23:36:50.221156 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.221139    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 23:36:50.221268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.221185    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 23:36:50.221268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.221211    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 23:36:50.221268 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.221244    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 23:36:50.223343 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.223319    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:50.249843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.249773    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.250535 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.250519    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.250611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.250551    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.250611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.250562    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.250611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.250597    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.256662 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.256649    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.256701 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.256669    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-115.ec2.internal\": node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.276945 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.276925    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.321693 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.321672    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"]
Sep 10 23:36:50.321765 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.321727    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.322442 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.322430    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.322484 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.322464    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.322484 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.322475    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.323526 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.323514    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.323614 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.323602    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.323680 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.323632    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.324234 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324220    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.324291 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324250    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.324291 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324260    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.324351 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324225    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.324351 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324316    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.324351 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.324329    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.325272 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.325257    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.325345 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.325281    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:36:50.325971 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.325955    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:36:50.326063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.325984    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:36:50.326063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.325997    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:36:50.346072 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.346048    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-115.ec2.internal\" not found" node="ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.350164 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.350148    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-115.ec2.internal\" not found" node="ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.377762 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.377742    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.378858 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.378843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.378900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.378865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.378900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.378882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a2bdc78eba34fc85c5ba40ae7a16c61-config\") pod \"kube-apiserver-proxy-ip-10-0-134-115.ec2.internal\" (UID: \"7a2bdc78eba34fc85c5ba40ae7a16c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.478754 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.478732    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.479854 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479839    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.479912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479863    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.479912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a2bdc78eba34fc85c5ba40ae7a16c61-config\") pod \"kube-apiserver-proxy-ip-10-0-134-115.ec2.internal\" (UID: \"7a2bdc78eba34fc85c5ba40ae7a16c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.479972 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a2bdc78eba34fc85c5ba40ae7a16c61-config\") pod \"kube-apiserver-proxy-ip-10-0-134-115.ec2.internal\" (UID: \"7a2bdc78eba34fc85c5ba40ae7a16c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.479972 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479941    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.479972 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.479951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aecc87e24059521f3b160ce9d5cc5a1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal\" (UID: \"aecc87e24059521f3b160ce9d5cc5a1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.579465 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.579407    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.647678 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.647648    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.652130 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.652107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:50.680329 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.680302    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.780790 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.780749    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.881289 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.881221    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.981799 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:50.981770    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:50.986583 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:50.986563    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:51.020716 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.020694    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 23:36:51.021127 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.020841    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 23:36:51.021127 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.020868    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 23:36:51.082441 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:51.082421    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:51.085881 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.085849    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 23:31:50 +0000 UTC" deadline="2028-02-17 07:28:28.104784204 +0000 UTC"
Sep 10 23:36:51.085881 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.085877    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12583h51m37.018909664s"
Sep 10 23:36:51.136263 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.136204    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:51.183236 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:51.183200    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:51.194347 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:51.194316    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaecc87e24059521f3b160ce9d5cc5a1c.slice/crio-6177dbeeac51ebe340514f8f6281a896089c1c062e296e4a201c75cc22181dc1 WatchSource:0}: Error finding container 6177dbeeac51ebe340514f8f6281a896089c1c062e296e4a201c75cc22181dc1: Status 404 returned error can't find the container with id 6177dbeeac51ebe340514f8f6281a896089c1c062e296e4a201c75cc22181dc1
Sep 10 23:36:51.194626 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:51.194604    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2bdc78eba34fc85c5ba40ae7a16c61.slice/crio-3cb793708a63b1c4cd53f2c775649efd058e4cdfd045e7f603a60775f29e3579 WatchSource:0}: Error finding container 3cb793708a63b1c4cd53f2c775649efd058e4cdfd045e7f603a60775f29e3579: Status 404 returned error can't find the container with id 3cb793708a63b1c4cd53f2c775649efd058e4cdfd045e7f603a60775f29e3579
Sep 10 23:36:51.198843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.198827    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:36:51.224564 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.224518    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal" event={"ID":"7a2bdc78eba34fc85c5ba40ae7a16c61","Type":"ContainerStarted","Data":"3cb793708a63b1c4cd53f2c775649efd058e4cdfd045e7f603a60775f29e3579"}
Sep 10 23:36:51.225586 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.225567    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal" event={"ID":"aecc87e24059521f3b160ce9d5cc5a1c","Type":"ContainerStarted","Data":"6177dbeeac51ebe340514f8f6281a896089c1c062e296e4a201c75cc22181dc1"}
Sep 10 23:36:51.283964 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:51.283933    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-115.ec2.internal\" not found"
Sep 10 23:36:51.320372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.319088    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:51.375825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.375795    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:51.387691 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.387632    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:36:51.388624 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.388610    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"
Sep 10 23:36:51.396710 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:51.396692    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:36:52.057089 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.057048    2575 apiserver.go:52] "Watching apiserver"
Sep 10 23:36:52.064011 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.063987    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 23:36:52.064398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.064372    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z","openshift-cluster-node-tuning-operator/tuned-k4fs7","openshift-image-registry/node-ca-6rwlz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal","openshift-multus/multus-additional-cni-plugins-vvdfz","openshift-multus/multus-gf86q","openshift-network-operator/iptables-alerter-w22k2","kube-system/global-pull-secret-syncer-2jmnx","kube-system/konnectivity-agent-rq7pl","openshift-dns/node-resolver-rcttb","openshift-multus/network-metrics-daemon-gq8xl","openshift-network-diagnostics/network-check-target-s2mbj","openshift-ovn-kubernetes/ovnkube-node-kskgn","kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal"]
Sep 10 23:36:52.066269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.066246    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.068042 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.068023    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.068502 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.068478    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.068629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.068611    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6lptc\""
Sep 10 23:36:52.068896 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.068878    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.070370 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.070350    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.070658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.070633    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 23:36:52.070745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.070691    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 23:36:52.070745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.070714    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.070868 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.070745    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 23:36:52.071288 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.071268    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ktc65\""
Sep 10 23:36:52.072483 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.072464    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.072601 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.072576    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:36:52.072866 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.072845    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.072948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.072925    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.073008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.072962    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 23:36:52.073065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.073051    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 23:36:52.073065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.073057    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.073193 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.073178    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hf66z\""
Sep 10 23:36:52.075496 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.075267    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.075916 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.075887    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 23:36:52.077445 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.077419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.077842 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.077822    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 23:36:52.077842 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.077833    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.078389 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.078372    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.078576 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.078552    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-28p27\""
Sep 10 23:36:52.078716 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.078701    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 23:36:52.078776 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.078727    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 23:36:52.079484 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.079462    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 23:36:52.079661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.079643    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tg2vs\""
Sep 10 23:36:52.080136 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.080117    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.082058 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.082037    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 23:36:52.082214 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.082191    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 23:36:52.082303 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.082270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kt4jq\""
Sep 10 23:36:52.082366 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.082320    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.087272 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.087249    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.087387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.087365    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.087458 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.087423    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.087675 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.087654    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 23:36:52.087938 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.087654    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.088119 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.088024    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kfrhm\""
Sep 10 23:36:52.089159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089137    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-systemd-units\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovn-node-metrics-cert\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.089398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnds\" (UniqueName: \"kubernetes.io/projected/0a93e61a-186f-4b2b-ba76-0e1498e04f67-kube-api-access-lnnds\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.089398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f409c393-b03d-4b8a-9250-3ce43782868a-hosts-file\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.089398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-systemd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089502 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-config\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089502 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvfxr\" (UniqueName: \"kubernetes.io/projected/2ef2ea39-f2e5-446d-9485-ae649a729cd3-kube-api-access-wvfxr\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.089502 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-cni-binary-copy\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-bin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-multus\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f409c393-b03d-4b8a-9250-3ce43782868a-tmp-dir\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.089658 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089648    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-kubelet\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089671    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-etc-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089677    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-node-log\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089701    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ef2ea39-f2e5-446d-9485-ae649a729cd3-host\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089745    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-socket-dir-parent\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-netns\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-conf-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6vqq\" (UniqueName: \"kubernetes.io/projected/f409c393-b03d-4b8a-9250-3ce43782868a-kube-api-access-d6vqq\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-slash\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.089891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-log-socket\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-netd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-dbus\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089948    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-system-cni-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.089973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ef2ea39-f2e5-446d-9485-ae649a729cd3-serviceca\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090045    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-kubelet-config\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090057    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-binary-copy\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-multus-certs\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090158    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090167    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dzgfc\""
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090083    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-etc-kubernetes\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plchn\" (UniqueName: \"kubernetes.io/projected/a9fec15c-141b-4faf-84f6-483fd55b9252-kube-api-access-plchn\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090295    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-netns\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-env-overrides\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.090397 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwk77\" (UniqueName: \"kubernetes.io/projected/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-kube-api-access-zwk77\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-k8s-cni-cncf-io\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-var-lib-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090420    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090444    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-bin\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-script-lib\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cnibin\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-kubelet\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-ovn\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090748    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090803    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k77kt\""
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.090993    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-system-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.091027    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-cnibin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.091071    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-hostroot\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.091097    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.091183    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:36:52.091331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.091223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-daemon-config\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.093574 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.093411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:52.093574 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.093467    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:36:52.120495 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.120472    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wt8b5"
Sep 10 23:36:52.128948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.128924    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wt8b5"
Sep 10 23:36:52.177202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.177179    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 23:36:52.191948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.191919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvfxr\" (UniqueName: \"kubernetes.io/projected/2ef2ea39-f2e5-446d-9485-ae649a729cd3-kube-api-access-wvfxr\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.191948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.191950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192131 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.191974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-bin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192186 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-multus\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192186 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-bin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192186 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-cni-multus\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192212    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-tuned\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-kubelet\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-node-log\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-kubelet\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-log-socket\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-node-log\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-dbus\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-log-socket\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-dbus\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-system-cni-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-etc-kubernetes\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-system-cni-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-host-slash\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-etc-kubernetes\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysconfig\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ef2ea39-f2e5-446d-9485-ae649a729cd3-serviceca\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192692    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-kubelet-config\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-binary-copy\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a27b058-3fdc-4b2a-9d93-467eb73078a9-kubelet-config\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-conf\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-sys\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.192873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-env-overrides\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192900    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-k8s-cni-cncf-io\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.192993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193001    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-k8s-cni-cncf-io\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193056    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193100    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-script-lib\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-kubelet\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ef2ea39-f2e5-446d-9485-ae649a729cd3-serviceca\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f667796-980e-4052-8a46-1731c04e6c7c-konnectivity-ca\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-binary-copy\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-var-lib-kubelet\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-system-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-daemon-config\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjbk2\" (UniqueName: \"kubernetes.io/projected/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-kube-api-access-vjbk2\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.193732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-env-overrides\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-system-cni-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-modprobe-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-systemd-units\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnds\" (UniqueName: \"kubernetes.io/projected/0a93e61a-186f-4b2b-ba76-0e1498e04f67-kube-api-access-lnnds\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-device-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-systemd-units\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f409c393-b03d-4b8a-9250-3ce43782868a-hosts-file\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-config\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-script-lib\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-cni-binary-copy\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.193845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f409c393-b03d-4b8a-9250-3ce43782868a-hosts-file\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.193853    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.194106    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:52.694065798 +0000 UTC m=+2.960062650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:52.194372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-daemon-config\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovnkube-config\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f667796-980e-4052-8a46-1731c04e6c7c-agent-certs\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-run\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5dt9\" (UniqueName: \"kubernetes.io/projected/a8648551-0047-47b8-a0af-291fdc3b8577-kube-api-access-l5dt9\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194674    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f409c393-b03d-4b8a-9250-3ce43782868a-tmp-dir\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-etc-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9fec15c-141b-4faf-84f6-483fd55b9252-cni-binary-copy\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ef2ea39-f2e5-446d-9485-ae649a729cd3-host\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-etc-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194758    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-socket-dir-parent\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194816    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ef2ea39-f2e5-446d-9485-ae649a729cd3-host\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-socket-dir-parent\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-netns\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-conf-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.194912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-kubernetes\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6vqq\" (UniqueName: \"kubernetes.io/projected/f409c393-b03d-4b8a-9250-3ce43782868a-kube-api-access-d6vqq\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.194989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-multus-conf-dir\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-slash\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f409c393-b03d-4b8a-9250-3ce43782868a-tmp-dir\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-netd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195046    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-netns\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195062    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195130    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-netd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-slash\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195167    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a93e61a-186f-4b2b-ba76-0e1498e04f67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-multus-certs\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plchn\" (UniqueName: \"kubernetes.io/projected/a9fec15c-141b-4faf-84f6-483fd55b9252-kube-api-access-plchn\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-var-lib-kubelet\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-host-run-multus-certs\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.195546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195291    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-tmp\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-lib-modules\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-netns\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwk77\" (UniqueName: \"kubernetes.io/projected/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-kube-api-access-zwk77\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-run-netns\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-iptables-alerter-script\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-var-lib-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-bin\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cnibin\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-var-lib-openvswitch\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-systemd\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-ovn\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a93e61a-186f-4b2b-ba76-0e1498e04f67-cnibin\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195630    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-cnibin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-cni-bin\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-hostroot\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.196355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-cnibin\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195678    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-ovn\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqv97\" (UniqueName: \"kubernetes.io/projected/b43d49ed-0952-42f7-8f5c-de0163530b07-kube-api-access-xqv97\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovn-node-metrics-cert\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195812    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9fec15c-141b-4faf-84f6-483fd55b9252-hostroot\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-host\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7f5\" (UniqueName: \"kubernetes.io/projected/91e91759-3e20-4227-ba15-f01548bf6fbf-kube-api-access-5z7f5\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-systemd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.195919    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.196005    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-run-systemd\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.197202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.196410    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 23:36:52.200683 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.200659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-ovn-node-metrics-cert\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.205907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.205887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvfxr\" (UniqueName: \"kubernetes.io/projected/2ef2ea39-f2e5-446d-9485-ae649a729cd3-kube-api-access-wvfxr\") pod \"node-ca-6rwlz\" (UID: \"2ef2ea39-f2e5-446d-9485-ae649a729cd3\") " pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.212648 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.212620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plchn\" (UniqueName: \"kubernetes.io/projected/a9fec15c-141b-4faf-84f6-483fd55b9252-kube-api-access-plchn\") pod \"multus-gf86q\" (UID: \"a9fec15c-141b-4faf-84f6-483fd55b9252\") " pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.213220 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.213196    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwk77\" (UniqueName: \"kubernetes.io/projected/20ce861e-2443-4f1b-bbeb-a20cfa62dfa0-kube-api-access-zwk77\") pod \"ovnkube-node-kskgn\" (UID: \"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.213307 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.213223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnds\" (UniqueName: \"kubernetes.io/projected/0a93e61a-186f-4b2b-ba76-0e1498e04f67-kube-api-access-lnnds\") pod \"multus-additional-cni-plugins-vvdfz\" (UID: \"0a93e61a-186f-4b2b-ba76-0e1498e04f67\") " pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.213627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.213596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6vqq\" (UniqueName: \"kubernetes.io/projected/f409c393-b03d-4b8a-9250-3ce43782868a-kube-api-access-d6vqq\") pod \"node-resolver-rcttb\" (UID: \"f409c393-b03d-4b8a-9250-3ce43782868a\") " pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.296916 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-var-lib-kubelet\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-tmp\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-lib-modules\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-iptables-alerter-script\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.296997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-systemd\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297005    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-var-lib-kubelet\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297024    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqv97\" (UniqueName: \"kubernetes.io/projected/b43d49ed-0952-42f7-8f5c-de0163530b07-kube-api-access-xqv97\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-systemd\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-host\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7f5\" (UniqueName: \"kubernetes.io/projected/91e91759-3e20-4227-ba15-f01548bf6fbf-kube-api-access-5z7f5\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-tuned\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-host-slash\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297284    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysconfig\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-conf\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297339    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-sys\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297372    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f667796-980e-4052-8a46-1731c04e6c7c-konnectivity-ca\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjbk2\" (UniqueName: \"kubernetes.io/projected/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-kube-api-access-vjbk2\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.297480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-modprobe-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-sys\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-device-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297564    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-device-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297547    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297372    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-lib-modules\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-iptables-alerter-script\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f667796-980e-4052-8a46-1731c04e6c7c-agent-certs\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-run\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5dt9\" (UniqueName: \"kubernetes.io/projected/a8648551-0047-47b8-a0af-291fdc3b8577-kube-api-access-l5dt9\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-host-slash\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297794    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-modprobe-d\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297812    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-kubernetes\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-host\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysconfig\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297869    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.297904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b43d49ed-0952-42f7-8f5c-de0163530b07-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.298011    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-kubernetes\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.298043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-sysctl-conf\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.298066    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91e91759-3e20-4227-ba15-f01548bf6fbf-run\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.298139    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.298215    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:52.798195586 +0000 UTC m=+3.064192438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:52.298850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.298556    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f667796-980e-4052-8a46-1731c04e6c7c-konnectivity-ca\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.300766 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.300729    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-tmp\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.301281 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.301250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f667796-980e-4052-8a46-1731c04e6c7c-agent-certs\") pod \"konnectivity-agent-rq7pl\" (UID: \"1f667796-980e-4052-8a46-1731c04e6c7c\") " pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.301376 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.301311    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91e91759-3e20-4227-ba15-f01548bf6fbf-etc-tuned\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.307987 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.307931    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:36:52.307987 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.307956    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:36:52.307987 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.307970    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:52.308221 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.308041    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:36:52.808024943 +0000 UTC m=+3.074021793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:52.309566 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.309542    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqv97\" (UniqueName: \"kubernetes.io/projected/b43d49ed-0952-42f7-8f5c-de0163530b07-kube-api-access-xqv97\") pod \"aws-ebs-csi-driver-node-qsv8z\" (UID: \"b43d49ed-0952-42f7-8f5c-de0163530b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.309674 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.309547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjbk2\" (UniqueName: \"kubernetes.io/projected/044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8-kube-api-access-vjbk2\") pod \"iptables-alerter-w22k2\" (UID: \"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8\") " pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.310707 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.310677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5dt9\" (UniqueName: \"kubernetes.io/projected/a8648551-0047-47b8-a0af-291fdc3b8577-kube-api-access-l5dt9\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.310707 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.310693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7f5\" (UniqueName: \"kubernetes.io/projected/91e91759-3e20-4227-ba15-f01548bf6fbf-kube-api-access-5z7f5\") pod \"tuned-k4fs7\" (UID: \"91e91759-3e20-4227-ba15-f01548bf6fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.343333 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.343310    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:52.378355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.378318    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcttb"
Sep 10 23:36:52.391345 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.391321    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rwlz"
Sep 10 23:36:52.399166 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.399147    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:36:52.405467 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.405445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vvdfz"
Sep 10 23:36:52.411070 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.411051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gf86q"
Sep 10 23:36:52.418017 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.417995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:36:52.424709 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.424692    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w22k2"
Sep 10 23:36:52.430499 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.430482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z"
Sep 10 23:36:52.436299 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.436279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4fs7"
Sep 10 23:36:52.701228 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.701135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:52.701407 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.701293    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:52.701407 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.701370    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:53.70134837 +0000 UTC m=+3.967345225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:52.802272 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.802238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:52.802438 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.802405    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:52.802512 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.802479    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:53.802457887 +0000 UTC m=+4.068454761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:52.902636 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:52.902597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:52.902828 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.902807    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:36:52.902869 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.902836    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:36:52.902869 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.902851    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:52.902945 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:52.902934    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:36:53.902897891 +0000 UTC m=+4.168894743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:53.100628 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.100596    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fec15c_141b_4faf_84f6_483fd55b9252.slice/crio-3ec475b78adced9d6874ace9f0b4ab6d1c85626edbd9e5f259beeec8ba47ccc4 WatchSource:0}: Error finding container 3ec475b78adced9d6874ace9f0b4ab6d1c85626edbd9e5f259beeec8ba47ccc4: Status 404 returned error can't find the container with id 3ec475b78adced9d6874ace9f0b4ab6d1c85626edbd9e5f259beeec8ba47ccc4
Sep 10 23:36:53.101592 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.101566    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef2ea39_f2e5_446d_9485_ae649a729cd3.slice/crio-9419963be1553d2493c0cfdddc54ef75538354c0da197028c0cab244df6cd984 WatchSource:0}: Error finding container 9419963be1553d2493c0cfdddc54ef75538354c0da197028c0cab244df6cd984: Status 404 returned error can't find the container with id 9419963be1553d2493c0cfdddc54ef75538354c0da197028c0cab244df6cd984
Sep 10 23:36:53.102286 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.102253    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ce861e_2443_4f1b_bbeb_a20cfa62dfa0.slice/crio-1191d812f4bcba79c1248e305121c80dd96fc35c005bf6ea8d98bd643e94ecc4 WatchSource:0}: Error finding container 1191d812f4bcba79c1248e305121c80dd96fc35c005bf6ea8d98bd643e94ecc4: Status 404 returned error can't find the container with id 1191d812f4bcba79c1248e305121c80dd96fc35c005bf6ea8d98bd643e94ecc4
Sep 10 23:36:53.105261 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.105242    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044fb8d8_0bb1_4ddd_bd11_bf5aae4385b8.slice/crio-c4093a3206a3760e163669e6fb184f0732635e8c80b81331ac6860f4279b2791 WatchSource:0}: Error finding container c4093a3206a3760e163669e6fb184f0732635e8c80b81331ac6860f4279b2791: Status 404 returned error can't find the container with id c4093a3206a3760e163669e6fb184f0732635e8c80b81331ac6860f4279b2791
Sep 10 23:36:53.128993 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.128975    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a93e61a_186f_4b2b_ba76_0e1498e04f67.slice/crio-f93d8d4ff774d90c4ff72f519ec4ce24a8113a22834c5c918ec1147949fe974e WatchSource:0}: Error finding container f93d8d4ff774d90c4ff72f519ec4ce24a8113a22834c5c918ec1147949fe974e: Status 404 returned error can't find the container with id f93d8d4ff774d90c4ff72f519ec4ce24a8113a22834c5c918ec1147949fe974e
Sep 10 23:36:53.129608 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.129585    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43d49ed_0952_42f7_8f5c_de0163530b07.slice/crio-feba8ba6a841f1b08544d4ab67ce79c548625b8d0a6e48efc70590c1934036f3 WatchSource:0}: Error finding container feba8ba6a841f1b08544d4ab67ce79c548625b8d0a6e48efc70590c1934036f3: Status 404 returned error can't find the container with id feba8ba6a841f1b08544d4ab67ce79c548625b8d0a6e48efc70590c1934036f3
Sep 10 23:36:53.131027 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.130989    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf409c393_b03d_4b8a_9250_3ce43782868a.slice/crio-0ff53eff631254a14165f6b5a201796d0e27e729fde48d9211e610065c4365f8 WatchSource:0}: Error finding container 0ff53eff631254a14165f6b5a201796d0e27e729fde48d9211e610065c4365f8: Status 404 returned error can't find the container with id 0ff53eff631254a14165f6b5a201796d0e27e729fde48d9211e610065c4365f8
Sep 10 23:36:53.131617 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.131562    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f667796_980e_4052_8a46_1731c04e6c7c.slice/crio-f727a4c961e484978f917574b53148541f47c37f927f6714a6eab80a91d4d484 WatchSource:0}: Error finding container f727a4c961e484978f917574b53148541f47c37f927f6714a6eab80a91d4d484: Status 404 returned error can't find the container with id f727a4c961e484978f917574b53148541f47c37f927f6714a6eab80a91d4d484
Sep 10 23:36:53.131782 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.131749    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 23:31:52 +0000 UTC" deadline="2028-02-22 17:51:28.669415779 +0000 UTC"
Sep 10 23:36:53.131855 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.131784    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12714h14m35.537636619s"
Sep 10 23:36:53.132470 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:36:53.132454    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e91759_3e20_4227_ba15_f01548bf6fbf.slice/crio-1e3a7e5d36a098d79b39d2c7e2eb73214e244ecec8e02b930e3346c2f1cf898b WatchSource:0}: Error finding container 1e3a7e5d36a098d79b39d2c7e2eb73214e244ecec8e02b930e3346c2f1cf898b: Status 404 returned error can't find the container with id 1e3a7e5d36a098d79b39d2c7e2eb73214e244ecec8e02b930e3346c2f1cf898b
Sep 10 23:36:53.221730 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.221695    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:53.221867 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.221840    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:36:53.221963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.221947    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:53.222077 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.222056    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:36:53.229930 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.229897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rq7pl" event={"ID":"1f667796-980e-4052-8a46-1731c04e6c7c","Type":"ContainerStarted","Data":"f727a4c961e484978f917574b53148541f47c37f927f6714a6eab80a91d4d484"}
Sep 10 23:36:53.231278 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.231250    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcttb" event={"ID":"f409c393-b03d-4b8a-9250-3ce43782868a","Type":"ContainerStarted","Data":"0ff53eff631254a14165f6b5a201796d0e27e729fde48d9211e610065c4365f8"}
Sep 10 23:36:53.232978 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.232950    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerStarted","Data":"f93d8d4ff774d90c4ff72f519ec4ce24a8113a22834c5c918ec1147949fe974e"}
Sep 10 23:36:53.234359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.234328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w22k2" event={"ID":"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8","Type":"ContainerStarted","Data":"c4093a3206a3760e163669e6fb184f0732635e8c80b81331ac6860f4279b2791"}
Sep 10 23:36:53.235624 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.235602    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rwlz" event={"ID":"2ef2ea39-f2e5-446d-9485-ae649a729cd3","Type":"ContainerStarted","Data":"9419963be1553d2493c0cfdddc54ef75538354c0da197028c0cab244df6cd984"}
Sep 10 23:36:53.236935 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.236886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"1191d812f4bcba79c1248e305121c80dd96fc35c005bf6ea8d98bd643e94ecc4"}
Sep 10 23:36:53.238224 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.238202    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gf86q" event={"ID":"a9fec15c-141b-4faf-84f6-483fd55b9252","Type":"ContainerStarted","Data":"3ec475b78adced9d6874ace9f0b4ab6d1c85626edbd9e5f259beeec8ba47ccc4"}
Sep 10 23:36:53.239435 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.239416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4fs7" event={"ID":"91e91759-3e20-4227-ba15-f01548bf6fbf","Type":"ContainerStarted","Data":"1e3a7e5d36a098d79b39d2c7e2eb73214e244ecec8e02b930e3346c2f1cf898b"}
Sep 10 23:36:53.240732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.240709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z" event={"ID":"b43d49ed-0952-42f7-8f5c-de0163530b07","Type":"ContainerStarted","Data":"feba8ba6a841f1b08544d4ab67ce79c548625b8d0a6e48efc70590c1934036f3"}
Sep 10 23:36:53.242820 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.242790    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal" event={"ID":"7a2bdc78eba34fc85c5ba40ae7a16c61","Type":"ContainerStarted","Data":"40d386033dd7e77659703c6d8d9a8432a3279af6de7b38c93c5ea33c28830730"}
Sep 10 23:36:53.257922 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.257868    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-115.ec2.internal" podStartSLOduration=1.257855108 podStartE2EDuration="1.257855108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:36:53.25765575 +0000 UTC m=+3.523652653" watchObservedRunningTime="2026-09-10 23:36:53.257855108 +0000 UTC m=+3.523851981"
Sep 10 23:36:53.357401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.357381    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:36:53.708892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.708800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:53.709081 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.709016    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:53.709148 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.709097    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:55.709074766 +0000 UTC m=+5.975071630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:53.809358 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.809317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:53.809557 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.809481    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:53.809651 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.809617    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:55.809595878 +0000 UTC m=+6.075592753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:53.910599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:53.910497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:53.910802 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.910733    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:36:53.910802 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.910759    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:36:53.910802 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.910773    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:53.910976 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:53.910832    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:36:55.910813621 +0000 UTC m=+6.176810482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:54.132899 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:54.132801    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 23:31:52 +0000 UTC" deadline="2028-06-22 13:41:38.159572386 +0000 UTC"
Sep 10 23:36:54.132899 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:54.132863    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15614h4m44.026713573s"
Sep 10 23:36:54.225299 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:54.224701    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:54.225299 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:54.224834    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:36:54.281034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:54.280387    2575 generic.go:356] "Generic (PLEG): container finished" podID="aecc87e24059521f3b160ce9d5cc5a1c" containerID="282241aab84f6173963d8b12a1ab04859e46f2ccf07ddbef5f7e6fe48054533e" exitCode=0
Sep 10 23:36:54.281034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:54.280465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal" event={"ID":"aecc87e24059521f3b160ce9d5cc5a1c","Type":"ContainerDied","Data":"282241aab84f6173963d8b12a1ab04859e46f2ccf07ddbef5f7e6fe48054533e"}
Sep 10 23:36:55.222330 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.222059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:55.222924 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.222455    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:36:55.222924 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.222281    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:55.223036 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.222937    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:36:55.296011 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.295971    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal" event={"ID":"aecc87e24059521f3b160ce9d5cc5a1c","Type":"ContainerStarted","Data":"24b85b930856cedb4c68fa43b70aabf06c2af061dd95a4b221a07e73373d703e"}
Sep 10 23:36:55.313565 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.313375    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-115.ec2.internal" podStartSLOduration=3.313345127 podStartE2EDuration="3.313345127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:36:55.311261691 +0000 UTC m=+5.577258565" watchObservedRunningTime="2026-09-10 23:36:55.313345127 +0000 UTC m=+5.579342005"
Sep 10 23:36:55.728488 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.728445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:55.728691 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.728624    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:55.728768 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.728693    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:59.728672947 +0000 UTC m=+9.994669800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:55.829742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.829703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:55.829983 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.829861    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:55.829983 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.829957    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:36:59.829935833 +0000 UTC m=+10.095932710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:55.930324 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:55.930284    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:55.930511 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.930496    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:36:55.930596 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.930518    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:36:55.930596 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.930550    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:55.930713 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:55.930612    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:36:59.930592183 +0000 UTC m=+10.196589051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:56.223300 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:56.222793    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:56.223300 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:56.222927    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:36:57.221476 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:57.221432    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:57.221689 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:57.221611    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:36:57.222027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:57.222007    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:57.222132 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:57.222108    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:36:58.224536 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:58.224002    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:58.224536 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:58.224148    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:36:59.221828 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:59.221750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:59.221970 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.221893    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:36:59.222252 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:59.222230    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:59.222343 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.222325    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:36:59.762972 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:59.762925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:36:59.763476 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.763166    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:59.763476 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.763261    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:07.763240011 +0000 UTC m=+18.029236862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:36:59.864024 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:59.863980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:36:59.864220 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.864134    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:59.864220 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.864206    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:07.86418546 +0000 UTC m=+18.130182325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:36:59.964711 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:36:59.964623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:36:59.964956 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.964825    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:36:59.964956 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.964848    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:36:59.964956 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.964863    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:36:59.964956 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:36:59.964925    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:37:07.964905954 +0000 UTC m=+18.230902812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:37:00.222751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:00.222711    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:00.222945 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:00.222841    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:01.222550 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:01.222335    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:01.222550 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:01.222458    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:01.223079 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:01.222877    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:01.223079 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:01.222992    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:02.222110 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:02.222067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:02.224044 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:02.224016    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:03.221412 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:03.221374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:03.221611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:03.221418    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:03.221611 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:03.221536    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:03.221730 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:03.221633    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:04.222123 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:04.222052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:04.222542 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:04.222185    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:05.221527 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:05.221486    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:05.221684 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:05.221601    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:05.221684 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:05.221629    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:05.221781 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:05.221696    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:06.221504 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:06.221462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:06.221941 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:06.221577    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:07.222564 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:07.222103    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:07.222564 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:07.222115    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:07.222564 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.222209    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:07.222564 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.222316    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:07.829626 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:07.829598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:07.829868 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.829728    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:37:07.829868 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.829781    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:23.82976584 +0000 UTC m=+34.095762698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:37:07.930298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:07.930265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:07.930458 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.930392    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:37:07.930524 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:07.930470    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:23.930448377 +0000 UTC m=+34.196445229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:37:08.031208 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:08.031172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:08.031369 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:08.031352    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:37:08.031416 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:08.031371    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:37:08.031416 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:08.031384    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:37:08.031512 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:08.031454    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:37:24.03143754 +0000 UTC m=+34.297434411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:37:08.222417 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:08.222319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:08.222600 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:08.222470    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:09.221588 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:09.221551    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:09.221775 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:09.221571    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:09.221775 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:09.221684    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:09.221898 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:09.221784    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:10.222817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:10.222780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:10.223272 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:10.222899    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:11.221954 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:11.221922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:11.221954 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:11.221963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:11.222196 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:11.222066    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:11.222250 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:11.222186    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:12.222664 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.222483    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:12.223549 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:12.222736    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:12.337882 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.337842    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rq7pl" event={"ID":"1f667796-980e-4052-8a46-1731c04e6c7c","Type":"ContainerStarted","Data":"56bf0550cc373a92d1019cd460cffe55c5a852de136e1c0a7e62519458b04730"}
Sep 10 23:37:12.339580 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.339553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcttb" event={"ID":"f409c393-b03d-4b8a-9250-3ce43782868a","Type":"ContainerStarted","Data":"cbe569e93b38833f15016b4a51ea2fcefbb70f370eb164e943faa218a3e0aded"}
Sep 10 23:37:12.341383 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.341361    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="030865f29eae655320412d4137dd4ddb829c62fa3af4d1b7e87bd940bba70d9e" exitCode=0
Sep 10 23:37:12.341533 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.341438    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"030865f29eae655320412d4137dd4ddb829c62fa3af4d1b7e87bd940bba70d9e"}
Sep 10 23:37:12.343527 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.343486    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rwlz" event={"ID":"2ef2ea39-f2e5-446d-9485-ae649a729cd3","Type":"ContainerStarted","Data":"ab20cded2acbb80736d3eef85dbcc084194bda8814de62b0f98cac4265993ae7"}
Sep 10 23:37:12.347048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347028    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"c0aab3964573e01df05d142a99983d4945466b7ee9e684e705e41348ce986ebd"}
Sep 10 23:37:12.347150 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347099    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"1d7d2957383470a6e0b149eee987289eb692d131f055ff00ff81441f1a0e8cbd"}
Sep 10 23:37:12.347209 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"2a7e753216a4215650af1389167134f7e941899f2a1c83ef258367cae05a2f15"}
Sep 10 23:37:12.347256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347239    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"079e703ec1a11639a1aa66551b1e2db37d0c29ff8c24b28b22649110c2042563"}
Sep 10 23:37:12.347314 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"66d6a7348a11761e5fd0007e98adb94bed78503c45f48b2f118b8eff5850af1c"}
Sep 10 23:37:12.347366 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.347339    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"2b47d8cca4e611179c7f20b6c733a5d2b1852dd6f17b6145426319ad5cc5fe94"}
Sep 10 23:37:12.349303 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.349254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gf86q" event={"ID":"a9fec15c-141b-4faf-84f6-483fd55b9252","Type":"ContainerStarted","Data":"39a6d46dbb2c017f629df02e4c75def9dba00ae14bda31e1c14cb3329b57d25f"}
Sep 10 23:37:12.350927 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.350903    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4fs7" event={"ID":"91e91759-3e20-4227-ba15-f01548bf6fbf","Type":"ContainerStarted","Data":"e4e269bf9542449a04b58cdd66d5c406ac75b45d30511712f7b0662fb8e8c22c"}
Sep 10 23:37:12.352426 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.352384    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rq7pl" podStartSLOduration=10.455313517 podStartE2EDuration="21.35237215s" totalImagesPullingTime="10.897058633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:12.351778287 +0000 UTC m=+22.617775162" watchObservedRunningTime="2026-09-10 23:37:12.35237215 +0000 UTC m=+22.618369021"
Sep 10 23:37:12.352714 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.352690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z" event={"ID":"b43d49ed-0952-42f7-8f5c-de0163530b07","Type":"ContainerStarted","Data":"bb807e9436c70171d5f9a767542402b2fc171ab0d08826885d86f673105965a9"}
Sep 10 23:37:12.370416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.370208    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rcttb" podStartSLOduration=3.149042634 podStartE2EDuration="21.370195462s" totalImagesPullingTime="18.221152828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:12.369598509 +0000 UTC m=+22.635595408" watchObservedRunningTime="2026-09-10 23:37:12.370195462 +0000 UTC m=+22.636192333"
Sep 10 23:37:12.390467 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.390427    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k4fs7" podStartSLOduration=3.153115933 podStartE2EDuration="21.390413548s" totalImagesPullingTime="18.237297615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:12.390071054 +0000 UTC m=+22.656067951" watchObservedRunningTime="2026-09-10 23:37:12.390413548 +0000 UTC m=+22.656410461"
Sep 10 23:37:12.404963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.404927    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6rwlz" podStartSLOduration=3.151689482 podStartE2EDuration="21.404916336s" totalImagesPullingTime="18.253226854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:12.404885712 +0000 UTC m=+22.670882602" watchObservedRunningTime="2026-09-10 23:37:12.404916336 +0000 UTC m=+22.670913206"
Sep 10 23:37:12.446779 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.446736    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gf86q" podStartSLOduration=3.141109063 podStartE2EDuration="21.446724183s" totalImagesPullingTime="18.30561512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:12.44617206 +0000 UTC m=+22.712168952" watchObservedRunningTime="2026-09-10 23:37:12.446724183 +0000 UTC m=+22.712721092"
Sep 10 23:37:12.800721 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:12.800689    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 23:37:13.162634 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.162544    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T23:37:12.800713851Z","UUID":"eacb5198-28ca-46ec-9a42-d5bf425ec99c","Handler":null,"Name":"","Endpoint":""}
Sep 10 23:37:13.164102 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.164079    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 23:37:13.164235 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.164115    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 23:37:13.222011 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.221986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:13.222189 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:13.222099    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:13.222311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.222290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:13.222464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:13.222437    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:13.357789 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.357755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w22k2" event={"ID":"044fb8d8-0bb1-4ddd-bd11-bf5aae4385b8","Type":"ContainerStarted","Data":"a1979f658935f60cd58c691185e284b92b3cf6b02f11495274b5685af57e5568"}
Sep 10 23:37:13.360137 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.360084    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z" event={"ID":"b43d49ed-0952-42f7-8f5c-de0163530b07","Type":"ContainerStarted","Data":"5aab1450dce083f128573d92f60bfb129b9312483ff618347224e6b9be7ed7dc"}
Sep 10 23:37:13.371966 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:13.371918    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w22k2" podStartSLOduration=4.142567446 podStartE2EDuration="22.371904432s" totalImagesPullingTime="18.229336986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:13.371592798 +0000 UTC m=+23.637589689" watchObservedRunningTime="2026-09-10 23:37:13.371904432 +0000 UTC m=+23.637901302"
Sep 10 23:37:14.221649 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:14.221623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:14.221797 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:14.221769    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:14.366943 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:14.366896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"8adadf8bea56166fbceb4d10b68f2146b43544fc7f5467ba01029d5e986cef64"}
Sep 10 23:37:14.369333 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:14.369305    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z" event={"ID":"b43d49ed-0952-42f7-8f5c-de0163530b07","Type":"ContainerStarted","Data":"b79b578964af4f7c0ce1ac611d29f6442d18cb0213aaabf76600078c5be35879"}
Sep 10 23:37:14.388964 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:14.388870    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv8z" podStartSLOduration=3.047493793 podStartE2EDuration="23.388855042s" totalImagesPullingTime="20.341361249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:14.388363808 +0000 UTC m=+24.654360695" watchObservedRunningTime="2026-09-10 23:37:14.388855042 +0000 UTC m=+24.654851910"
Sep 10 23:37:15.221609 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:15.221566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:15.221609 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:15.221608    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:15.221840 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:15.221697    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:15.221892 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:15.221834    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:16.222306 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:16.222265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:16.222724 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:16.222459    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:16.964456 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:16.964280    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:37:16.964927 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:16.964910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:37:17.221455 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.221378    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:17.221455 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.221410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:17.221644 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:17.221519    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:17.221644 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:17.221618    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:17.378201 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.378165    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="8ac504917fad08d5ba1005c3877e0e6412d5a98b57a36fb988c93f3e84accf97" exitCode=0
Sep 10 23:37:17.378921 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.378247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"8ac504917fad08d5ba1005c3877e0e6412d5a98b57a36fb988c93f3e84accf97"}
Sep 10 23:37:17.382656 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.382632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" event={"ID":"20ce861e-2443-4f1b-bbeb-a20cfa62dfa0","Type":"ContainerStarted","Data":"6547643666eb036887d9c3d05f4a10dd794a8875ae28c997b664acb643e1c425"}
Sep 10 23:37:17.382895 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.382872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:37:17.383373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.383351    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rq7pl"
Sep 10 23:37:17.426939 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:17.426896    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn" podStartSLOduration=8.132868335 podStartE2EDuration="26.426881939s" totalImagesPullingTime="18.294013604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:17.425475654 +0000 UTC m=+27.691472525" watchObservedRunningTime="2026-09-10 23:37:17.426881939 +0000 UTC m=+27.692878809"
Sep 10 23:37:18.222375 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.222342    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:18.222565 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:18.222449    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:18.387651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.387397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerStarted","Data":"39f1db4894b37bfcf95579ef398fcd834593cca2a69c60dabb90cc47c33e44da"}
Sep 10 23:37:18.387974 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.387828    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:18.387974 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.387888    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:18.388090 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.388047    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:18.404258 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.404237    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:18.404351 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.404341    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:18.503980 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.503873    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2mbj"]
Sep 10 23:37:18.503980 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.503958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:18.504148 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:18.504033    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:18.510643 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.510607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2jmnx"]
Sep 10 23:37:18.510788 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.510714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:18.510848 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:18.510805    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:18.511310 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.511246    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gq8xl"]
Sep 10 23:37:18.511430 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:18.511357    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:18.511511 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:18.511470    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:19.391624 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:19.391587    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="39f1db4894b37bfcf95579ef398fcd834593cca2a69c60dabb90cc47c33e44da" exitCode=0
Sep 10 23:37:19.392074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:19.391675    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"39f1db4894b37bfcf95579ef398fcd834593cca2a69c60dabb90cc47c33e44da"}
Sep 10 23:37:20.223069 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:20.223036    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:20.223240 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:20.223118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:20.223240 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:20.223126    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:20.223240 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:20.223227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:20.223358 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:20.223228    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:20.223358 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:20.223305    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:21.399376 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:21.399206    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="55184d9c7caaef5af09592cfd8c5116bf559dc90a06ab1ef14c1da867cace36d" exitCode=0
Sep 10 23:37:21.399376 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:21.399262    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"55184d9c7caaef5af09592cfd8c5116bf559dc90a06ab1ef14c1da867cace36d"}
Sep 10 23:37:22.222068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:22.222033    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:22.222250 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:22.222047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:22.222250 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:22.222156    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:22.222250 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:22.222179    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2jmnx" podUID="4a27b058-3fdc-4b2a-9d93-467eb73078a9"
Sep 10 23:37:22.222405 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:22.222273    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2mbj" podUID="b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b"
Sep 10 23:37:22.222405 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:22.222369    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8xl" podUID="a8648551-0047-47b8-a0af-291fdc3b8577"
Sep 10 23:37:23.850704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:23.850665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:23.851140 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:23.850835    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:37:23.851140 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:23.850902    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret podName:4a27b058-3fdc-4b2a-9d93-467eb73078a9 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:55.850887858 +0000 UTC m=+66.116884707 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret") pod "global-pull-secret-syncer-2jmnx" (UID: "4a27b058-3fdc-4b2a-9d93-467eb73078a9") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:37:23.952048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:23.952012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:23.952223 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:23.952200    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:37:23.952302 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:23.952289    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs podName:a8648551-0047-47b8-a0af-291fdc3b8577 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:55.952267662 +0000 UTC m=+66.218264530 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs") pod "network-metrics-daemon-gq8xl" (UID: "a8648551-0047-47b8-a0af-291fdc3b8577") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:37:23.988595 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:23.988563    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-115.ec2.internal" event="NodeReady"
Sep 10 23:37:23.988766 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:23.988717    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 23:37:24.032914 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.032882    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n4wp8"]
Sep 10 23:37:24.038177 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.038142    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xf4f5"]
Sep 10 23:37:24.038339 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.038221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.040569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.040545    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 23:37:24.040569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.040560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6vpbd\""
Sep 10 23:37:24.040841 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.040816    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 23:37:24.041390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.041374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.043419 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.043398    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 23:37:24.043553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.043449    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 23:37:24.043620 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.043559    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 23:37:24.043620 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.043575    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-45clb\""
Sep 10 23:37:24.045282 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.045251    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf4f5"]
Sep 10 23:37:24.049423 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.048788    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4wp8"]
Sep 10 23:37:24.052554 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.052531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:24.052714 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.052676    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:37:24.052714 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.052701    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:37:24.052714 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.052712    2575 projected.go:196] Error preparing data for projected volume kube-api-access-k9kpp for pod openshift-network-diagnostics/network-check-target-s2mbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:37:24.052928 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.052765    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp podName:b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b nodeName:}" failed. No retries permitted until 2026-09-10 23:37:56.052748241 +0000 UTC m=+66.318745104 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9kpp" (UniqueName: "kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp") pod "network-check-target-s2mbj" (UID: "b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:37:24.153218 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2889n\" (UniqueName: \"kubernetes.io/projected/058cc32b-8fa4-47de-a597-88d349640ecb-kube-api-access-2889n\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.153218 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/058cc32b-8fa4-47de-a597-88d349640ecb-config-volume\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.153218 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.153514 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz797\" (UniqueName: \"kubernetes.io/projected/d53c74b5-5763-455e-8e99-a6446c62421b-kube-api-access-xz797\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.153514 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53c74b5-5763-455e-8e99-a6446c62421b-cert\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.153514 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.153338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/058cc32b-8fa4-47de-a597-88d349640ecb-tmp-dir\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.221509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.221456    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:24.222079 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.222056    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:24.222307 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.222286    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:24.224057 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224035    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 23:37:24.224057 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224048    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 23:37:24.224206 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cql7l\""
Sep 10 23:37:24.224258 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224224    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 23:37:24.224310 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224296    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 23:37:24.224411 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.224379    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xgqvn\""
Sep 10 23:37:24.254162 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2889n\" (UniqueName: \"kubernetes.io/projected/058cc32b-8fa4-47de-a597-88d349640ecb-kube-api-access-2889n\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.254162 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/058cc32b-8fa4-47de-a597-88d349640ecb-config-volume\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.254344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.254344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254201    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz797\" (UniqueName: \"kubernetes.io/projected/d53c74b5-5763-455e-8e99-a6446c62421b-kube-api-access-xz797\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.254344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53c74b5-5763-455e-8e99-a6446c62421b-cert\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.254344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/058cc32b-8fa4-47de-a597-88d349640ecb-tmp-dir\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.254344 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.254316    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:24.254560 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.254382    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:24.75436128 +0000 UTC m=+35.020358146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:24.254784 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/058cc32b-8fa4-47de-a597-88d349640ecb-config-volume\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.254982 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.254962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/058cc32b-8fa4-47de-a597-88d349640ecb-tmp-dir\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.259039 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.259021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53c74b5-5763-455e-8e99-a6446c62421b-cert\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.263202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.263146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz797\" (UniqueName: \"kubernetes.io/projected/d53c74b5-5763-455e-8e99-a6446c62421b-kube-api-access-xz797\") pod \"ingress-canary-xf4f5\" (UID: \"d53c74b5-5763-455e-8e99-a6446c62421b\") " pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.263303 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.263232    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2889n\" (UniqueName: \"kubernetes.io/projected/058cc32b-8fa4-47de-a597-88d349640ecb-kube-api-access-2889n\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.358576 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.358542    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf4f5"
Sep 10 23:37:24.758868 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:24.758827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:24.759063 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.758960    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:24.759063 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:24.759019    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:25.759004003 +0000 UTC m=+36.025000865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:25.644686 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:25.644440    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf4f5"]
Sep 10 23:37:25.652577 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:25.652551    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53c74b5_5763_455e_8e99_a6446c62421b.slice/crio-357df9cef5a820464c7bb6312b96696711b3886d1e879c119e9f6769bc88ee1e WatchSource:0}: Error finding container 357df9cef5a820464c7bb6312b96696711b3886d1e879c119e9f6769bc88ee1e: Status 404 returned error can't find the container with id 357df9cef5a820464c7bb6312b96696711b3886d1e879c119e9f6769bc88ee1e
Sep 10 23:37:25.766677 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:25.766643    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:25.766821 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:25.766801    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:25.766879 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:25.766870    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:27.766853423 +0000 UTC m=+38.032850290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:26.413435 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:26.413398    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf4f5" event={"ID":"d53c74b5-5763-455e-8e99-a6446c62421b","Type":"ContainerStarted","Data":"357df9cef5a820464c7bb6312b96696711b3886d1e879c119e9f6769bc88ee1e"}
Sep 10 23:37:26.416832 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:26.416793    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="70b56d806294b8f1e27ee4010eab06e3c5b35bfe09005efc23a8986c45a752bd" exitCode=0
Sep 10 23:37:26.416969 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:26.416839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"70b56d806294b8f1e27ee4010eab06e3c5b35bfe09005efc23a8986c45a752bd"}
Sep 10 23:37:27.356195 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.356161    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"]
Sep 10 23:37:27.368704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.368666    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"]
Sep 10 23:37:27.368867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.368794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"
Sep 10 23:37:27.372026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.371997    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 23:37:27.372026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.372011    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:37:27.372239 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.372111    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4nwsf\""
Sep 10 23:37:27.374380 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.374360    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"]
Sep 10 23:37:27.374796 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.374777    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.379571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.379237    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5p96d\""
Sep 10 23:37:27.379571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.379387    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 23:37:27.379571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.379506    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:37:27.379907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.379886    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 23:37:27.383714 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.383692    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 23:37:27.384521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.384479    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"]
Sep 10 23:37:27.384617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.384542    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"]
Sep 10 23:37:27.384617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.384565    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"]
Sep 10 23:37:27.384617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.384599    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.388128 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.388108    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 23:37:27.388452 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.388436    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 23:37:27.388908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.388889    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7c8gx\""
Sep 10 23:37:27.389109 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.389096    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 23:37:27.389147 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.389105    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:37:27.423361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.423316    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a93e61a-186f-4b2b-ba76-0e1498e04f67" containerID="1a3509182440acb1522f54ded3c1a2884781beb52a448d8e86749c52d28657a3" exitCode=0
Sep 10 23:37:27.423559 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.423391    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerDied","Data":"1a3509182440acb1522f54ded3c1a2884781beb52a448d8e86749c52d28657a3"}
Sep 10 23:37:27.480354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480310    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrmn7\" (UniqueName: \"kubernetes.io/projected/33bc7525-926c-411b-9d7f-f7bc2378d10a-kube-api-access-zrmn7\") pod \"volume-data-source-validator-86695b7cc7-tb4bz\" (UID: \"33bc7525-926c-411b-9d7f-f7bc2378d10a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"
Sep 10 23:37:27.480354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-serving-cert\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.480599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480394    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kzmf\" (UniqueName: \"kubernetes.io/projected/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-kube-api-access-7kzmf\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.480599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8802c81-0116-4bb9-abbe-f2da5c7cb762-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.480599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8802c81-0116-4bb9-abbe-f2da5c7cb762-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.480599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfgdr\" (UniqueName: \"kubernetes.io/projected/a8802c81-0116-4bb9-abbe-f2da5c7cb762-kube-api-access-qfgdr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.480772 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.480612    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-config\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.566518 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.566062    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-55546df84-vmltb"]
Sep 10 23:37:27.581373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-serving-cert\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.581579 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kzmf\" (UniqueName: \"kubernetes.io/projected/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-kube-api-access-7kzmf\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.581870 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8802c81-0116-4bb9-abbe-f2da5c7cb762-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.581971 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8802c81-0116-4bb9-abbe-f2da5c7cb762-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.581971 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581903    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfgdr\" (UniqueName: \"kubernetes.io/projected/a8802c81-0116-4bb9-abbe-f2da5c7cb762-kube-api-access-qfgdr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.581971 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-config\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.582121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.581981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrmn7\" (UniqueName: \"kubernetes.io/projected/33bc7525-926c-411b-9d7f-f7bc2378d10a-kube-api-access-zrmn7\") pod \"volume-data-source-validator-86695b7cc7-tb4bz\" (UID: \"33bc7525-926c-411b-9d7f-f7bc2378d10a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"
Sep 10 23:37:27.582636 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.582613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-config\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.582748 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.582718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8802c81-0116-4bb9-abbe-f2da5c7cb762-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.585439 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.585415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8802c81-0116-4bb9-abbe-f2da5c7cb762-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.585562 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.585414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-serving-cert\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.586069 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.586051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.589615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.589593    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 23:37:27.589615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.589610    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 23:37:27.589867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.589853    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 23:37:27.590095 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.590062    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tpbtv\""
Sep 10 23:37:27.590298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.590281    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 23:37:27.590502 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.590476    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 23:37:27.590727 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.590710    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 23:37:27.590911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.590855    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55546df84-vmltb"]
Sep 10 23:37:27.600437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.600413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfgdr\" (UniqueName: \"kubernetes.io/projected/a8802c81-0116-4bb9-abbe-f2da5c7cb762-kube-api-access-qfgdr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b8v84\" (UID: \"a8802c81-0116-4bb9-abbe-f2da5c7cb762\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.600895 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.600869    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kzmf\" (UniqueName: \"kubernetes.io/projected/ba1d9df4-9515-4ce2-95f8-323bdd3c2b05-kube-api-access-7kzmf\") pod \"service-ca-operator-cf7c776b7-7xpbd\" (UID: \"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.602178 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.602157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrmn7\" (UniqueName: \"kubernetes.io/projected/33bc7525-926c-411b-9d7f-f7bc2378d10a-kube-api-access-zrmn7\") pod \"volume-data-source-validator-86695b7cc7-tb4bz\" (UID: \"33bc7525-926c-411b-9d7f-f7bc2378d10a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"
Sep 10 23:37:27.683052 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683013    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"
Sep 10 23:37:27.683235 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-stats-auth\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.683303 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.683364 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.683418 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-default-certificate\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.683418 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.683403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd8h2\" (UniqueName: \"kubernetes.io/projected/8c541dc9-c33b-483a-8502-7dcdff60c7fe-kube-api-access-vd8h2\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.692377 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.692348    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"
Sep 10 23:37:27.699545 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.699520    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-default-certificate\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd8h2\" (UniqueName: \"kubernetes.io/projected/8c541dc9-c33b-483a-8502-7dcdff60c7fe-kube-api-access-vd8h2\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-stats-auth\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.785346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:27.785518    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:28.285478589 +0000 UTC m=+38.551475460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:27.786200    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:27.786293    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:31.786274054 +0000 UTC m=+42.052270920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:27.786200    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:27.786756 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:27.786342    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:28.286332279 +0000 UTC m=+38.552329129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:27.790318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.790259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-stats-auth\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.792533 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.792505    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-default-certificate\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:27.795443 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:27.795418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd8h2\" (UniqueName: \"kubernetes.io/projected/8c541dc9-c33b-483a-8502-7dcdff60c7fe-kube-api-access-vd8h2\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:28.050119 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.049935    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd"]
Sep 10 23:37:28.053572 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:28.053544    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1d9df4_9515_4ce2_95f8_323bdd3c2b05.slice/crio-540a2b9cd60b56b3273ab1e17e88ef4097087034077d3298802ccb8a1fdb2414 WatchSource:0}: Error finding container 540a2b9cd60b56b3273ab1e17e88ef4097087034077d3298802ccb8a1fdb2414: Status 404 returned error can't find the container with id 540a2b9cd60b56b3273ab1e17e88ef4097087034077d3298802ccb8a1fdb2414
Sep 10 23:37:28.057627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.057564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84"]
Sep 10 23:37:28.060713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.060650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz"]
Sep 10 23:37:28.061426 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:28.061395    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8802c81_0116_4bb9_abbe_f2da5c7cb762.slice/crio-494efc06e034c52e2dd68798610b1dd7f9a5ea2e68c21ee27da842535f1d213a WatchSource:0}: Error finding container 494efc06e034c52e2dd68798610b1dd7f9a5ea2e68c21ee27da842535f1d213a: Status 404 returned error can't find the container with id 494efc06e034c52e2dd68798610b1dd7f9a5ea2e68c21ee27da842535f1d213a
Sep 10 23:37:28.066129 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:28.066109    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bc7525_926c_411b_9d7f_f7bc2378d10a.slice/crio-87ac51d172061897b360167063e6a60d0097c56d1cd84221c31caaf42acb2978 WatchSource:0}: Error finding container 87ac51d172061897b360167063e6a60d0097c56d1cd84221c31caaf42acb2978: Status 404 returned error can't find the container with id 87ac51d172061897b360167063e6a60d0097c56d1cd84221c31caaf42acb2978
Sep 10 23:37:28.290266 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.290232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:28.290434 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.290296    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:28.290434 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:28.290409    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:29.29039063 +0000 UTC m=+39.556387482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:28.290434 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:28.290413    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:28.290621 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:28.290450    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:29.290441303 +0000 UTC m=+39.556438152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:28.427443 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.427407    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz" event={"ID":"33bc7525-926c-411b-9d7f-f7bc2378d10a","Type":"ContainerStarted","Data":"87ac51d172061897b360167063e6a60d0097c56d1cd84221c31caaf42acb2978"}
Sep 10 23:37:28.431071 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.431041    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" event={"ID":"0a93e61a-186f-4b2b-ba76-0e1498e04f67","Type":"ContainerStarted","Data":"e6f562d8a457734b0d21aca183f4451cf7f928825901ef8bdb36b17fd092cc3f"}
Sep 10 23:37:28.432248 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.432223    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84" event={"ID":"a8802c81-0116-4bb9-abbe-f2da5c7cb762","Type":"ContainerStarted","Data":"494efc06e034c52e2dd68798610b1dd7f9a5ea2e68c21ee27da842535f1d213a"}
Sep 10 23:37:28.433319 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.433301    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd" event={"ID":"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05","Type":"ContainerStarted","Data":"540a2b9cd60b56b3273ab1e17e88ef4097087034077d3298802ccb8a1fdb2414"}
Sep 10 23:37:28.434881 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.434860    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf4f5" event={"ID":"d53c74b5-5763-455e-8e99-a6446c62421b","Type":"ContainerStarted","Data":"13835af06b0d8e9c0bfe005dfd3395af8940c56f09896eb541b5b03dfce115ae"}
Sep 10 23:37:28.449997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.449955    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vvdfz" podStartSLOduration=9.210106827 podStartE2EDuration="37.449942645s" totalImagesPullingTime="28.239835818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:28.449014982 +0000 UTC m=+38.715011853" watchObservedRunningTime="2026-09-10 23:37:28.449942645 +0000 UTC m=+38.715939513"
Sep 10 23:37:28.465114 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:28.465076    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xf4f5" podStartSLOduration=2.175130018 podStartE2EDuration="4.465062026s" totalImagesPullingTime="2.289932008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:28.464412297 +0000 UTC m=+38.730409188" watchObservedRunningTime="2026-09-10 23:37:28.465062026 +0000 UTC m=+38.731058897"
Sep 10 23:37:29.299196 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:29.299155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:29.299368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:29.299264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:29.299446 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:29.299419    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:31.299397724 +0000 UTC m=+41.565394780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:29.299920 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:29.299888    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:29.300061 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:29.299951    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:31.299935305 +0000 UTC m=+41.565932174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:31.314734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.314699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:31.315222 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.314807    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:31.315222 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:31.314847    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:31.315222 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:31.314924    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:35.314901304 +0000 UTC m=+45.580898170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:31.315222 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:31.314945    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:35.314934181 +0000 UTC m=+45.580931036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:31.444130 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.444090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84" event={"ID":"a8802c81-0116-4bb9-abbe-f2da5c7cb762","Type":"ContainerStarted","Data":"1fb4281a099ec3d6f9bc5edb017537494edd9c06fb53cb5c5f2845402ad45e5a"}
Sep 10 23:37:31.445733 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.445652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd" event={"ID":"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05","Type":"ContainerStarted","Data":"7571ae2a416dff9dd12cecf04605f8c33cc9f99f01a9751ca65854e2c1f64a99"}
Sep 10 23:37:31.447209 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.447187    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz" event={"ID":"33bc7525-926c-411b-9d7f-f7bc2378d10a","Type":"ContainerStarted","Data":"88cb6153dd07240072798adcd44b4ede1bde8a5dc35ee81e05c86b4ff77708bc"}
Sep 10 23:37:31.458977 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.458906    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84" podStartSLOduration=1.302579003 podStartE2EDuration="4.458891996s" totalImagesPullingTime="3.156312993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:31.457735122 +0000 UTC m=+41.723732012" watchObservedRunningTime="2026-09-10 23:37:31.458891996 +0000 UTC m=+41.724888863"
Sep 10 23:37:31.485204 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.485153    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tb4bz" podStartSLOduration=1.359432838 podStartE2EDuration="4.485136024s" totalImagesPullingTime="3.125703186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:31.484433768 +0000 UTC m=+41.750430658" watchObservedRunningTime="2026-09-10 23:37:31.485136024 +0000 UTC m=+41.751132894"
Sep 10 23:37:31.485474 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.485454    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd" podStartSLOduration=1.3266405940000001 podStartE2EDuration="4.485447782s" totalImagesPullingTime="3.158807188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:31.471063788 +0000 UTC m=+41.737060660" watchObservedRunningTime="2026-09-10 23:37:31.485447782 +0000 UTC m=+41.751444653"
Sep 10 23:37:31.819391 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:31.819354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:31.819609 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:31.819541    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:31.819670 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:31.819618    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:39.8196021 +0000 UTC m=+50.085598948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:32.368788 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.368759    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"]
Sep 10 23:37:32.401885 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.401856    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"]
Sep 10 23:37:32.402069 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.401967    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"
Sep 10 23:37:32.404596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.404572    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 23:37:32.404738 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.404641    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kwlwv\""
Sep 10 23:37:32.404738 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.404665    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 23:37:32.424614 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.424575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8bj7\" (UniqueName: \"kubernetes.io/projected/0455cf93-2914-4f8d-aae4-0d2c6494f4fe-kube-api-access-p8bj7\") pod \"migrator-8b4d697c5-qhfjg\" (UID: \"0455cf93-2914-4f8d-aae4-0d2c6494f4fe\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"
Sep 10 23:37:32.525528 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.525473    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8bj7\" (UniqueName: \"kubernetes.io/projected/0455cf93-2914-4f8d-aae4-0d2c6494f4fe-kube-api-access-p8bj7\") pod \"migrator-8b4d697c5-qhfjg\" (UID: \"0455cf93-2914-4f8d-aae4-0d2c6494f4fe\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"
Sep 10 23:37:32.537558 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.537530    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8bj7\" (UniqueName: \"kubernetes.io/projected/0455cf93-2914-4f8d-aae4-0d2c6494f4fe-kube-api-access-p8bj7\") pod \"migrator-8b4d697c5-qhfjg\" (UID: \"0455cf93-2914-4f8d-aae4-0d2c6494f4fe\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"
Sep 10 23:37:32.712315 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.712233    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"
Sep 10 23:37:32.911243 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.911213    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rrxm2"]
Sep 10 23:37:32.944274 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.944238    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rrxm2"]
Sep 10 23:37:32.944436 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.944359    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:32.947193 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.947168    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 23:37:32.947326 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.947170    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 23:37:32.947426 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.947411    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 23:37:32.947542 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.947525    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4wv72\""
Sep 10 23:37:32.948222 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:32.948205    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 23:37:33.006260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.006186    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg"]
Sep 10 23:37:33.009473 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:33.009442    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0455cf93_2914_4f8d_aae4_0d2c6494f4fe.slice/crio-0f23f355fdf2fdbcb4054b38e1660ea509c2ecd1c75c2c13513be01951c677b3 WatchSource:0}: Error finding container 0f23f355fdf2fdbcb4054b38e1660ea509c2ecd1c75c2c13513be01951c677b3: Status 404 returned error can't find the container with id 0f23f355fdf2fdbcb4054b38e1660ea509c2ecd1c75c2c13513be01951c677b3
Sep 10 23:37:33.029601 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.029577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/59546425-503d-457a-b4c9-fd5c40740f81-crio-socket\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.029706 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.029608    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/59546425-503d-457a-b4c9-fd5c40740f81-data-volume\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.029706 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.029669    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl7xc\" (UniqueName: \"kubernetes.io/projected/59546425-503d-457a-b4c9-fd5c40740f81-kube-api-access-gl7xc\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.029798 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.029707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.029798 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.029733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130151 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/59546425-503d-457a-b4c9-fd5c40740f81-crio-socket\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130151 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/59546425-503d-457a-b4c9-fd5c40740f81-data-volume\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl7xc\" (UniqueName: \"kubernetes.io/projected/59546425-503d-457a-b4c9-fd5c40740f81-kube-api-access-gl7xc\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130342    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/59546425-503d-457a-b4c9-fd5c40740f81-crio-socket\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.130482    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.130556    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:33.630541533 +0000 UTC m=+43.896538381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : configmap "kube-rbac-proxy" not found
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.130555    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/59546425-503d-457a-b4c9-fd5c40740f81-data-volume\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.130572    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:33.130625 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.130625    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:33.630609048 +0000 UTC m=+43.896605903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:33.138879 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.138859    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl7xc\" (UniqueName: \"kubernetes.io/projected/59546425-503d-457a-b4c9-fd5c40740f81-kube-api-access-gl7xc\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.452668 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.452635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg" event={"ID":"0455cf93-2914-4f8d-aae4-0d2c6494f4fe","Type":"ContainerStarted","Data":"0f23f355fdf2fdbcb4054b38e1660ea509c2ecd1c75c2c13513be01951c677b3"}
Sep 10 23:37:33.633051 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.633015    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.633051 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:33.633053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:33.633290 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.633130    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:37:33.633290 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.633178    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:33.633290 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.633197    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:34.633179627 +0000 UTC m=+44.899176492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : configmap "kube-rbac-proxy" not found
Sep 10 23:37:33.633290 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:33.633218    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:34.63320733 +0000 UTC m=+44.899204179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:34.640482 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:34.640456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:34.640874 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:34.640522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:34.640874 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:34.640571    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:37:34.640874 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:34.640628    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:36.640613819 +0000 UTC m=+46.906610685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : configmap "kube-rbac-proxy" not found
Sep 10 23:37:34.640874 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:34.640641    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:34.640874 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:34.640685    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:36.64066805 +0000 UTC m=+46.906664898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:35.345437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.345397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:35.345690 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.345524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:35.345690 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:35.345575    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:35.345690 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:35.345640    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:43.345625148 +0000 UTC m=+53.611621996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:35.345690 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:35.345653    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:43.345647313 +0000 UTC m=+53.611644162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:35.460594 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.460560    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg" event={"ID":"0455cf93-2914-4f8d-aae4-0d2c6494f4fe","Type":"ContainerStarted","Data":"8b5d4f0de4ceabcf2e01dde258248b7de7a1667a2d45a02c0dc77250acbb8ecf"}
Sep 10 23:37:35.460740 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.460611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg" event={"ID":"0455cf93-2914-4f8d-aae4-0d2c6494f4fe","Type":"ContainerStarted","Data":"740a8902730e7d4cf2938c4ca5765632c40c2070f26c6c39b1f93b9e7ac78ae4"}
Sep 10 23:37:35.483602 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.483553    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qhfjg" podStartSLOduration=2.025061611 podStartE2EDuration="3.483531991s" totalImagesPullingTime="1.45847038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:35.482048098 +0000 UTC m=+45.748044968" watchObservedRunningTime="2026-09-10 23:37:35.483531991 +0000 UTC m=+45.749528866"
Sep 10 23:37:35.880738 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:35.880710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcttb_f409c393-b03d-4b8a-9250-3ce43782868a/dns-node-resolver/0.log"
Sep 10 23:37:36.478194 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:36.478160    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rwlz_2ef2ea39-f2e5-446d-9485-ae649a729cd3/node-ca/0.log"
Sep 10 23:37:36.656298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:36.656227    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:36.656298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:36.656261    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:36.656464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:36.656359    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:37:36.656464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:36.656413    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:36.656464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:36.656430    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:40.656413552 +0000 UTC m=+50.922410421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : configmap "kube-rbac-proxy" not found
Sep 10 23:37:36.656464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:36.656451    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:40.656440101 +0000 UTC m=+50.922436950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:37.678889 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:37.678851    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf4f5_d53c74b5-5763-455e-8e99-a6446c62421b/serve-healthcheck-canary/0.log"
Sep 10 23:37:37.878616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:37.878591    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qhfjg_0455cf93-2914-4f8d-aae4-0d2c6494f4fe/migrator/0.log"
Sep 10 23:37:38.078925 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:38.078904    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qhfjg_0455cf93-2914-4f8d-aae4-0d2c6494f4fe/graceful-termination/0.log"
Sep 10 23:37:38.280032 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:38.280007    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-b8v84_a8802c81-0116-4bb9-abbe-f2da5c7cb762/kube-storage-version-migrator-operator/0.log"
Sep 10 23:37:39.881146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:39.881115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:39.881533 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:39.881233    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:37:39.881533 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:39.881285    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls podName:058cc32b-8fa4-47de-a597-88d349640ecb nodeName:}" failed. No retries permitted until 2026-09-10 23:37:55.88127188 +0000 UTC m=+66.147268728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls") pod "dns-default-n4wp8" (UID: "058cc32b-8fa4-47de-a597-88d349640ecb") : secret "dns-default-metrics-tls" not found
Sep 10 23:37:40.688589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:40.688549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:40.688589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:40.688590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:40.688808 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:40.688693    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:37:40.688808 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:40.688735    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:40.688808 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:40.688767    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:48.688748363 +0000 UTC m=+58.954745231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : configmap "kube-rbac-proxy" not found
Sep 10 23:37:40.688808 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:40.688782    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:37:48.688775808 +0000 UTC m=+58.954772659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:43.407733 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:43.407693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:43.408209 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:43.407753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:43.408209 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:43.407865    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:59.407839442 +0000 UTC m=+69.673836291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : configmap references non-existent config key: service-ca.crt
Sep 10 23:37:43.408209 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:43.407867    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:37:43.408209 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:43.407905    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs podName:8c541dc9-c33b-483a-8502-7dcdff60c7fe nodeName:}" failed. No retries permitted until 2026-09-10 23:37:59.407896908 +0000 UTC m=+69.673893757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs") pod "router-default-55546df84-vmltb" (UID: "8c541dc9-c33b-483a-8502-7dcdff60c7fe") : secret "router-metrics-certs-default" not found
Sep 10 23:37:48.752121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:48.752084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:48.752121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:48.752122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:48.752549 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:48.752259    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:48.752549 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:37:48.752312    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls podName:59546425-503d-457a-b4c9-fd5c40740f81 nodeName:}" failed. No retries permitted until 2026-09-10 23:38:04.752299379 +0000 UTC m=+75.018296232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rrxm2" (UID: "59546425-503d-457a-b4c9-fd5c40740f81") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:37:48.752679 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:48.752659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/59546425-503d-457a-b4c9-fd5c40740f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:37:50.404911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:50.404881    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kskgn"
Sep 10 23:37:55.909917 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:55.909880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:55.910313 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:55.909953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:55.912529 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:55.912469    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/058cc32b-8fa4-47de-a597-88d349640ecb-metrics-tls\") pod \"dns-default-n4wp8\" (UID: \"058cc32b-8fa4-47de-a597-88d349640ecb\") " pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:55.912646 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:55.912576    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 23:37:55.922647 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:55.922616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a27b058-3fdc-4b2a-9d93-467eb73078a9-original-pull-secret\") pod \"global-pull-secret-syncer-2jmnx\" (UID: \"4a27b058-3fdc-4b2a-9d93-467eb73078a9\") " pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:56.010659 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.010619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:56.013329 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.013308    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 23:37:56.023682 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.023656    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8648551-0047-47b8-a0af-291fdc3b8577-metrics-certs\") pod \"network-metrics-daemon-gq8xl\" (UID: \"a8648551-0047-47b8-a0af-291fdc3b8577\") " pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:56.041343 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.041316    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cql7l\""
Sep 10 23:37:56.044352 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.044334    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2jmnx"
Sep 10 23:37:56.049472 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.049447    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8xl"
Sep 10 23:37:56.111070 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.111042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:56.113822 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.113799    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 23:37:56.124029 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.124007    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 23:37:56.134391 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.134345    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9kpp\" (UniqueName: \"kubernetes.io/projected/b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b-kube-api-access-k9kpp\") pod \"network-check-target-s2mbj\" (UID: \"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b\") " pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:56.154359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.154331    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6vpbd\""
Sep 10 23:37:56.162809 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.162431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:56.335709 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.335677    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xgqvn\""
Sep 10 23:37:56.343784 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.343758    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:37:56.362347 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.362210    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2jmnx"]
Sep 10 23:37:56.365514 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:56.365384    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a27b058_3fdc_4b2a_9d93_467eb73078a9.slice/crio-993adbcffe889f6f07c1f5ad4cc6de98307db97cf76ec4aba341b8286ca11b92 WatchSource:0}: Error finding container 993adbcffe889f6f07c1f5ad4cc6de98307db97cf76ec4aba341b8286ca11b92: Status 404 returned error can't find the container with id 993adbcffe889f6f07c1f5ad4cc6de98307db97cf76ec4aba341b8286ca11b92
Sep 10 23:37:56.386391 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.386364    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gq8xl"]
Sep 10 23:37:56.389789 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:56.389761    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8648551_0047_47b8_a0af_291fdc3b8577.slice/crio-6fc6c8ef539f3b4d0c1c7279c7c1f0750f784b0632ded5c3dde178c0adada981 WatchSource:0}: Error finding container 6fc6c8ef539f3b4d0c1c7279c7c1f0750f784b0632ded5c3dde178c0adada981: Status 404 returned error can't find the container with id 6fc6c8ef539f3b4d0c1c7279c7c1f0750f784b0632ded5c3dde178c0adada981
Sep 10 23:37:56.452732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.452711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4wp8"]
Sep 10 23:37:56.453912 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:56.453882    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058cc32b_8fa4_47de_a597_88d349640ecb.slice/crio-12c1c7e411b2edbb3d0d5afd10ea43dd071bffe1aa4b1e197bedd5efe2dbe73b WatchSource:0}: Error finding container 12c1c7e411b2edbb3d0d5afd10ea43dd071bffe1aa4b1e197bedd5efe2dbe73b: Status 404 returned error can't find the container with id 12c1c7e411b2edbb3d0d5afd10ea43dd071bffe1aa4b1e197bedd5efe2dbe73b
Sep 10 23:37:56.528453 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.528415    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2jmnx" event={"ID":"4a27b058-3fdc-4b2a-9d93-467eb73078a9","Type":"ContainerStarted","Data":"993adbcffe889f6f07c1f5ad4cc6de98307db97cf76ec4aba341b8286ca11b92"}
Sep 10 23:37:56.529558 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.529526    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4wp8" event={"ID":"058cc32b-8fa4-47de-a597-88d349640ecb","Type":"ContainerStarted","Data":"12c1c7e411b2edbb3d0d5afd10ea43dd071bffe1aa4b1e197bedd5efe2dbe73b"}
Sep 10 23:37:56.530683 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.530663    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8xl" event={"ID":"a8648551-0047-47b8-a0af-291fdc3b8577","Type":"ContainerStarted","Data":"6fc6c8ef539f3b4d0c1c7279c7c1f0750f784b0632ded5c3dde178c0adada981"}
Sep 10 23:37:56.639914 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:56.639870    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2mbj"]
Sep 10 23:37:56.644680 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:37:56.644645    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3809c1c_9ed7_4c06_8e09_fc2aa4b5c55b.slice/crio-1a683ffd0395db8709919d91e59d56cf42b36056ce25dfd4679d698253e9fb96 WatchSource:0}: Error finding container 1a683ffd0395db8709919d91e59d56cf42b36056ce25dfd4679d698253e9fb96: Status 404 returned error can't find the container with id 1a683ffd0395db8709919d91e59d56cf42b36056ce25dfd4679d698253e9fb96
Sep 10 23:37:57.537042 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:57.536944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2mbj" event={"ID":"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b","Type":"ContainerStarted","Data":"1a683ffd0395db8709919d91e59d56cf42b36056ce25dfd4679d698253e9fb96"}
Sep 10 23:37:59.441409 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.441378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:59.441856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.441451    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:59.442280 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.442172    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c541dc9-c33b-483a-8502-7dcdff60c7fe-service-ca-bundle\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:59.444232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.444210    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c541dc9-c33b-483a-8502-7dcdff60c7fe-metrics-certs\") pod \"router-default-55546df84-vmltb\" (UID: \"8c541dc9-c33b-483a-8502-7dcdff60c7fe\") " pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:37:59.547329 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.547292    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4wp8" event={"ID":"058cc32b-8fa4-47de-a597-88d349640ecb","Type":"ContainerStarted","Data":"6db1a620f9ac4932fb609aa8d0a2762ae0d735ee5b3b4253a0887b50a2007c0a"}
Sep 10 23:37:59.547535 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.547355    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4wp8" event={"ID":"058cc32b-8fa4-47de-a597-88d349640ecb","Type":"ContainerStarted","Data":"91d70327b85f51f56706162e2df26fb8815482f55ccbc40b5ec8c3c8f6710365"}
Sep 10 23:37:59.547535 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.547418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:37:59.549429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.549399    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8xl" event={"ID":"a8648551-0047-47b8-a0af-291fdc3b8577","Type":"ContainerStarted","Data":"2ea10092388c4b8891361b11bfe6795eb2cd926eb12e77c85415d27715dfc8ee"}
Sep 10 23:37:59.549558 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.549453    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8xl" event={"ID":"a8648551-0047-47b8-a0af-291fdc3b8577","Type":"ContainerStarted","Data":"8c70c53690d4a6f0511b9d3ee58cf4dc8d6ee3b5f8f844890062338aaa00da59"}
Sep 10 23:37:59.567041 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.566993    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n4wp8" podStartSLOduration=33.358575613 podStartE2EDuration="35.566974535s" totalImagesPullingTime="2.208398922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:59.565706755 +0000 UTC m=+69.831703627" watchObservedRunningTime="2026-09-10 23:37:59.566974535 +0000 UTC m=+69.832971408"
Sep 10 23:37:59.582051 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.581997    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gq8xl" podStartSLOduration=66.443895228 podStartE2EDuration="1m8.581979481s" totalImagesPullingTime="2.138084253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:37:59.581203679 +0000 UTC m=+69.847200567" watchObservedRunningTime="2026-09-10 23:37:59.581979481 +0000 UTC m=+69.847976354"
Sep 10 23:37:59.720118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.720042    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tpbtv\""
Sep 10 23:37:59.728853 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:37:59.728825    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:38:01.559516 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:01.559196    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2mbj" event={"ID":"b3809c1c-9ed7-4c06-8e09-fc2aa4b5c55b","Type":"ContainerStarted","Data":"74afba7a1712426d168463e017ff190ab4d22e33252c5ad9df6dc5b8b19f4c74"}
Sep 10 23:38:01.559516 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:01.559481    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:38:01.574008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:01.573968    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s2mbj" podStartSLOduration=65.753104018 podStartE2EDuration="1m10.573953846s" totalImagesPullingTime="4.820849828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:01.573000818 +0000 UTC m=+71.838997704" watchObservedRunningTime="2026-09-10 23:38:01.573953846 +0000 UTC m=+71.839950755"
Sep 10 23:38:01.730961 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:01.730887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55546df84-vmltb"]
Sep 10 23:38:01.734128 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:01.734103    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c541dc9_c33b_483a_8502_7dcdff60c7fe.slice/crio-2a2e01ad98af2f25a259bbc4a6a2b4138258067712899de6290d0d7f53bef668 WatchSource:0}: Error finding container 2a2e01ad98af2f25a259bbc4a6a2b4138258067712899de6290d0d7f53bef668: Status 404 returned error can't find the container with id 2a2e01ad98af2f25a259bbc4a6a2b4138258067712899de6290d0d7f53bef668
Sep 10 23:38:02.564372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.564333    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2jmnx" event={"ID":"4a27b058-3fdc-4b2a-9d93-467eb73078a9","Type":"ContainerStarted","Data":"94b0a087be910c3bd4e911e277a0eb0d7593b327b80e22d26301eddf12e067cd"}
Sep 10 23:38:02.565912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.565888    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55546df84-vmltb" event={"ID":"8c541dc9-c33b-483a-8502-7dcdff60c7fe","Type":"ContainerStarted","Data":"8e1173a7f2a248b0ea8410d6581073841786cc677736d7fe1a20131949eb09b7"}
Sep 10 23:38:02.566027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.565921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55546df84-vmltb" event={"ID":"8c541dc9-c33b-483a-8502-7dcdff60c7fe","Type":"ContainerStarted","Data":"2a2e01ad98af2f25a259bbc4a6a2b4138258067712899de6290d0d7f53bef668"}
Sep 10 23:38:02.578683 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.578647    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2jmnx" podStartSLOduration=66.481355309 podStartE2EDuration="1m11.578633651s" totalImagesPullingTime="5.097278342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:02.578190206 +0000 UTC m=+72.844187092" watchObservedRunningTime="2026-09-10 23:38:02.578633651 +0000 UTC m=+72.844630575"
Sep 10 23:38:02.594840 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.594803    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-55546df84-vmltb" podStartSLOduration=35.594790913 podStartE2EDuration="35.594790913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:02.594034444 +0000 UTC m=+72.860031350" watchObservedRunningTime="2026-09-10 23:38:02.594790913 +0000 UTC m=+72.860787817"
Sep 10 23:38:02.730049 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.730019    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:38:02.732764 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:02.732744    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:38:03.157135 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.157105    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"]
Sep 10 23:38:03.159254 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.159238    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:03.161704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.161686    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 23:38:03.162118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.162098    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j7xjw\""
Sep 10 23:38:03.169412 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.169394    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"]
Sep 10 23:38:03.270897 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.270866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-j54lv\" (UID: \"fefb9e1b-a0d9-4ed8-a964-83f1587276fd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:03.371159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.371125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-j54lv\" (UID: \"fefb9e1b-a0d9-4ed8-a964-83f1587276fd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:03.371303 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:03.371258    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 23:38:03.371342 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:03.371319    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates podName:fefb9e1b-a0d9-4ed8-a964-83f1587276fd nodeName:}" failed. No retries permitted until 2026-09-10 23:38:03.871302372 +0000 UTC m=+74.137299222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-j54lv" (UID: "fefb9e1b-a0d9-4ed8-a964-83f1587276fd") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 23:38:03.569567 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.569534    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:38:03.570823 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.570807    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-55546df84-vmltb"
Sep 10 23:38:03.875994 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.875903    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-j54lv\" (UID: \"fefb9e1b-a0d9-4ed8-a964-83f1587276fd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:03.878332 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:03.878304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fefb9e1b-a0d9-4ed8-a964-83f1587276fd-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-j54lv\" (UID: \"fefb9e1b-a0d9-4ed8-a964-83f1587276fd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:04.068757 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:04.068715    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:04.335381 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:04.335355    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"]
Sep 10 23:38:04.338722 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:04.338698    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefb9e1b_a0d9_4ed8_a964_83f1587276fd.slice/crio-c5c882375bae7c0a7f57d910c6ce4c8b55461b61e05dedaeb194d6fa5473f9ad WatchSource:0}: Error finding container c5c882375bae7c0a7f57d910c6ce4c8b55461b61e05dedaeb194d6fa5473f9ad: Status 404 returned error can't find the container with id c5c882375bae7c0a7f57d910c6ce4c8b55461b61e05dedaeb194d6fa5473f9ad
Sep 10 23:38:04.573582 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:04.573540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv" event={"ID":"fefb9e1b-a0d9-4ed8-a964-83f1587276fd","Type":"ContainerStarted","Data":"c5c882375bae7c0a7f57d910c6ce4c8b55461b61e05dedaeb194d6fa5473f9ad"}
Sep 10 23:38:04.782402 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:04.782368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:38:04.784800 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:04.784782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/59546425-503d-457a-b4c9-fd5c40740f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rrxm2\" (UID: \"59546425-503d-457a-b4c9-fd5c40740f81\") " pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:38:05.056211 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:05.056138    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4wv72\""
Sep 10 23:38:05.063993 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:05.063968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rrxm2"
Sep 10 23:38:05.346112 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:05.346027    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rrxm2"]
Sep 10 23:38:05.427792 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:05.427756    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59546425_503d_457a_b4c9_fd5c40740f81.slice/crio-bc4d2559245bc0ec05b71f706a3ca4830adb4065da74a0fe37f402b2265c76a2 WatchSource:0}: Error finding container bc4d2559245bc0ec05b71f706a3ca4830adb4065da74a0fe37f402b2265c76a2: Status 404 returned error can't find the container with id bc4d2559245bc0ec05b71f706a3ca4830adb4065da74a0fe37f402b2265c76a2
Sep 10 23:38:05.578251 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:05.578226    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rrxm2" event={"ID":"59546425-503d-457a-b4c9-fd5c40740f81","Type":"ContainerStarted","Data":"b70e438b8cc338d3cf018eddfe0d5091f7edc7a69026014af8013dc63438a13e"}
Sep 10 23:38:05.578601 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:05.578278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rrxm2" event={"ID":"59546425-503d-457a-b4c9-fd5c40740f81","Type":"ContainerStarted","Data":"bc4d2559245bc0ec05b71f706a3ca4830adb4065da74a0fe37f402b2265c76a2"}
Sep 10 23:38:06.583919 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:06.583863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv" event={"ID":"fefb9e1b-a0d9-4ed8-a964-83f1587276fd","Type":"ContainerStarted","Data":"2ca688106d063ac4e264523f5cd454e133ab54ad252bc965648095aa8d4afe7d"}
Sep 10 23:38:06.584378 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:06.584028    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:06.586297 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:06.586268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rrxm2" event={"ID":"59546425-503d-457a-b4c9-fd5c40740f81","Type":"ContainerStarted","Data":"ff20f6b4db7eff0f6e08ff4e663ffceb3bd029d9331e973c6d4b8f6330df5031"}
Sep 10 23:38:06.589839 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:06.589804    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv"
Sep 10 23:38:06.599507 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:06.599440    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-j54lv" podStartSLOduration=2.197264691 podStartE2EDuration="3.599421983s" totalImagesPullingTime="1.402157292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:06.598632295 +0000 UTC m=+76.864629188" watchObservedRunningTime="2026-09-10 23:38:06.599421983 +0000 UTC m=+76.865418855"
Sep 10 23:38:07.220253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.220217    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"]
Sep 10 23:38:07.223088 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.223063    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.225615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.225592    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 23:38:07.226410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.226387    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 23:38:07.226685 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.226664    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 23:38:07.226685 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.226680    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 23:38:07.226953 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.226935    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w75nl\""
Sep 10 23:38:07.227172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.227032    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 23:38:07.235125 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.235102    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"]
Sep 10 23:38:07.304087 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.304068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b006c8-0e17-4735-aa0c-df035c5224ae-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.304148 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.304099    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47gg\" (UniqueName: \"kubernetes.io/projected/89b006c8-0e17-4735-aa0c-df035c5224ae-kube-api-access-c47gg\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.304183 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.304146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.304183 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.304167    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.404443 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.404418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.404552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.404448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.404552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.404478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b006c8-0e17-4735-aa0c-df035c5224ae-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.404552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.404521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47gg\" (UniqueName: \"kubernetes.io/projected/89b006c8-0e17-4735-aa0c-df035c5224ae-kube-api-access-c47gg\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.405198 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.405176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b006c8-0e17-4735-aa0c-df035c5224ae-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.406829 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.406804    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.407059 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.407039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89b006c8-0e17-4735-aa0c-df035c5224ae-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.414796 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.414777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47gg\" (UniqueName: \"kubernetes.io/projected/89b006c8-0e17-4735-aa0c-df035c5224ae-kube-api-access-c47gg\") pod \"prometheus-operator-68bcd7f987-kffdk\" (UID: \"89b006c8-0e17-4735-aa0c-df035c5224ae\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.534997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.534970    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"
Sep 10 23:38:07.592560 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.592528    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rrxm2" event={"ID":"59546425-503d-457a-b4c9-fd5c40740f81","Type":"ContainerStarted","Data":"09a9a2223f58753ff394ac23e46104c2c66ab75e4e565dbc2b31d839a924220e"}
Sep 10 23:38:07.609152 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.609106    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rrxm2" podStartSLOduration=33.893148393 podStartE2EDuration="35.609088312s" totalImagesPullingTime="1.715939919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:37:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:07.608606784 +0000 UTC m=+77.874603676" watchObservedRunningTime="2026-09-10 23:38:07.609088312 +0000 UTC m=+77.875085182"
Sep 10 23:38:07.802485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:07.802416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kffdk"]
Sep 10 23:38:07.805336 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:07.805311    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b006c8_0e17_4735_aa0c_df035c5224ae.slice/crio-5280d35e01f05610258ead5c81eab31f6397fc4c33addc93951e199394d38f72 WatchSource:0}: Error finding container 5280d35e01f05610258ead5c81eab31f6397fc4c33addc93951e199394d38f72: Status 404 returned error can't find the container with id 5280d35e01f05610258ead5c81eab31f6397fc4c33addc93951e199394d38f72
Sep 10 23:38:08.598421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:08.598388    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk" event={"ID":"89b006c8-0e17-4735-aa0c-df035c5224ae","Type":"ContainerStarted","Data":"5280d35e01f05610258ead5c81eab31f6397fc4c33addc93951e199394d38f72"}
Sep 10 23:38:09.556606 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:09.556579    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n4wp8"
Sep 10 23:38:09.604779 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:09.604749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk" event={"ID":"89b006c8-0e17-4735-aa0c-df035c5224ae","Type":"ContainerStarted","Data":"8d85325eb892fda837344adca37295b2e2b670362705ce6a90250a01d3809b8c"}
Sep 10 23:38:09.605165 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:09.604806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk" event={"ID":"89b006c8-0e17-4735-aa0c-df035c5224ae","Type":"ContainerStarted","Data":"92f42c6a47aac6ecf6c84a26c1b00e3d3868836edba1e6157fb1a8fd15e3d317"}
Sep 10 23:38:09.620957 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:09.620910    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kffdk" podStartSLOduration=1.48277392 podStartE2EDuration="2.620896706s" totalImagesPullingTime="1.138122786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:09.619285645 +0000 UTC m=+79.885282539" watchObservedRunningTime="2026-09-10 23:38:09.620896706 +0000 UTC m=+79.886893576"
Sep 10 23:38:11.571885 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.571779    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"]
Sep 10 23:38:11.575149 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.575134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.578426 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.578403    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 23:38:11.578543 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.578454    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 23:38:11.578543 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.578412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4qt9c\""
Sep 10 23:38:11.578751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.578737    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 23:38:11.591232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.591210    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"]
Sep 10 23:38:11.594193 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.594172    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-264xv"]
Sep 10 23:38:11.597579 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.597561    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.599569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.599551    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 23:38:11.599569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.599557    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 23:38:11.599822 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.599808    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 23:38:11.599927 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.599913    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v8rxg\""
Sep 10 23:38:11.739251 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739219    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739297    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2lb2\" (UniqueName: \"kubernetes.io/projected/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-api-access-m2lb2\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhg6\" (UniqueName: \"kubernetes.io/projected/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-kube-api-access-svhg6\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-sys\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-tls\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739455    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739517    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739544    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-wtmp\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-metrics-client-ca\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-root\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.739739 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.739630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-textfile\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.840817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.840729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-wtmp\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.840817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.840765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-metrics-client-ca\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.840817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.840785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.840983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-root\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-root\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-textfile\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841217 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.840983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-wtmp\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841217 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841217 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841161    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841217 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841193    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2lb2\" (UniqueName: \"kubernetes.io/projected/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-api-access-m2lb2\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841248    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhg6\" (UniqueName: \"kubernetes.io/projected/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-kube-api-access-svhg6\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-sys\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841282    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-textfile\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-tls\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:11.841369    2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 23:38:11.841413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-metrics-client-ca\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841821 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:11.841430    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls podName:6c1df7c8-4c23-45d9-b873-865cce8c4bfa nodeName:}" failed. No retries permitted until 2026-09-10 23:38:12.341410361 +0000 UTC m=+82.607407211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-pv7d9" (UID: "6c1df7c8-4c23-45d9-b873-865cce8c4bfa") : secret "kube-state-metrics-tls" not found
Sep 10 23:38:11.841821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-sys\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.841821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841458    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.841821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.842238 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.841976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.842238 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.842049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.843849 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.843823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.844045 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.844028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.844183 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.844168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-node-exporter-tls\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.851804 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.851780    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2lb2\" (UniqueName: \"kubernetes.io/projected/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-api-access-m2lb2\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:11.852174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.852155    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhg6\" (UniqueName: \"kubernetes.io/projected/a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb-kube-api-access-svhg6\") pod \"node-exporter-264xv\" (UID: \"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb\") " pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.906372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:11.906337    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-264xv"
Sep 10 23:38:11.918048 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:11.918019    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda456052d_0d2a_4fe0_91cb_ae6ac0c68dfb.slice/crio-6e1a51b0cc4fadf34dfc24c4bce695138b7a1dbac2987ea58faed9b66ba75137 WatchSource:0}: Error finding container 6e1a51b0cc4fadf34dfc24c4bce695138b7a1dbac2987ea58faed9b66ba75137: Status 404 returned error can't find the container with id 6e1a51b0cc4fadf34dfc24c4bce695138b7a1dbac2987ea58faed9b66ba75137
Sep 10 23:38:12.344918 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.344850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:12.347836 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.347806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c1df7c8-4c23-45d9-b873-865cce8c4bfa-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-pv7d9\" (UID: \"6c1df7c8-4c23-45d9-b873-865cce8c4bfa\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:12.484766 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.484730    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"
Sep 10 23:38:12.617385 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.617287    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-264xv" event={"ID":"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb","Type":"ContainerStarted","Data":"6e1a51b0cc4fadf34dfc24c4bce695138b7a1dbac2987ea58faed9b66ba75137"}
Sep 10 23:38:12.665286 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.665254    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:38:12.671685 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.671660    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.674573 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.674548    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 23:38:12.674712 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.674688    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.674949    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.674979    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.675028    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.674980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.675031    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2lqsh\""
Sep 10 23:38:12.675212 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.675123    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 23:38:12.675520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.675327    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 23:38:12.675520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.675346    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 23:38:12.687199 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.687178    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:38:12.748266 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748454 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748454 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748454 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748454 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcs6q\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748594    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.748745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.748737    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.777265 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.777239    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9"]
Sep 10 23:38:12.798848 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:12.798822    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1df7c8_4c23_45d9_b873_865cce8c4bfa.slice/crio-11e122376da99ff0228771b418a8754b2dc0e10df4dbfb19d83f8583d032eba2 WatchSource:0}: Error finding container 11e122376da99ff0228771b418a8754b2dc0e10df4dbfb19d83f8583d032eba2: Status 404 returned error can't find the container with id 11e122376da99ff0228771b418a8754b2dc0e10df4dbfb19d83f8583d032eba2
Sep 10 23:38:12.835594 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.835332    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-8clcw"]
Sep 10 23:38:12.841696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.841676    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:12.843987 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.843969    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kfmsq\""
Sep 10 23:38:12.844089 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.844006    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 23:38:12.844089 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.844006    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 23:38:12.846969 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.846950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-8clcw"]
Sep 10 23:38:12.850050 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850058    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcs6q\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850432 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850676 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:12.850628    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 10 23:38:12.850725 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:38:12.850696    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls podName:66c71a90-1b02-4dda-bbad-151de0610645 nodeName:}" failed. No retries permitted until 2026-09-10 23:38:13.350674007 +0000 UTC m=+83.616670861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "66c71a90-1b02-4dda-bbad-151de0610645") : secret "alertmanager-main-tls" not found
Sep 10 23:38:12.850876 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850816    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.850945 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.851003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.851003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.850980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.851104 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.851020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.852078 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.851155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.852078 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.851794    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.852723 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.852699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.854142 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.854071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.854715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.854669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.855235 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.855216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.855944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.855611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.855944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.855895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.855944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.855934    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.855944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.855937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.856285 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.856269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.860598 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.860569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcs6q\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:12.952669 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:12.952546    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsklr\" (UniqueName: \"kubernetes.io/projected/c666dc5c-ed83-4620-a4a9-954acdf30026-kube-api-access-fsklr\") pod \"downloads-76f99584f4-8clcw\" (UID: \"c666dc5c-ed83-4620-a4a9-954acdf30026\") " pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:13.053171 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.053133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsklr\" (UniqueName: \"kubernetes.io/projected/c666dc5c-ed83-4620-a4a9-954acdf30026-kube-api-access-fsklr\") pod \"downloads-76f99584f4-8clcw\" (UID: \"c666dc5c-ed83-4620-a4a9-954acdf30026\") " pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:13.061081 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.061055    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsklr\" (UniqueName: \"kubernetes.io/projected/c666dc5c-ed83-4620-a4a9-954acdf30026-kube-api-access-fsklr\") pod \"downloads-76f99584f4-8clcw\" (UID: \"c666dc5c-ed83-4620-a4a9-954acdf30026\") " pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:13.195827 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.195748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:13.356447 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.356412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:13.359830 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.359800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:13.485089 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.484996    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-8clcw"]
Sep 10 23:38:13.489247 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:13.489147    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc666dc5c_ed83_4620_a4a9_954acdf30026.slice/crio-5325b9277d349b70c94a65ede3f9783b9704081c483cbb69212f941b22e5b6c3 WatchSource:0}: Error finding container 5325b9277d349b70c94a65ede3f9783b9704081c483cbb69212f941b22e5b6c3: Status 404 returned error can't find the container with id 5325b9277d349b70c94a65ede3f9783b9704081c483cbb69212f941b22e5b6c3
Sep 10 23:38:13.583584 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.583553    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:38:13.622758 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.622710    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9" event={"ID":"6c1df7c8-4c23-45d9-b873-865cce8c4bfa","Type":"ContainerStarted","Data":"11e122376da99ff0228771b418a8754b2dc0e10df4dbfb19d83f8583d032eba2"}
Sep 10 23:38:13.624824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.624790    2575 generic.go:356] "Generic (PLEG): container finished" podID="a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb" containerID="b3c5f312b391824db867da196d64fb380c8cf71cc798b419826dcf382dbccaef" exitCode=0
Sep 10 23:38:13.624961 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.624875    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-264xv" event={"ID":"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb","Type":"ContainerDied","Data":"b3c5f312b391824db867da196d64fb380c8cf71cc798b419826dcf382dbccaef"}
Sep 10 23:38:13.626291 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.626265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-8clcw" event={"ID":"c666dc5c-ed83-4620-a4a9-954acdf30026","Type":"ContainerStarted","Data":"5325b9277d349b70c94a65ede3f9783b9704081c483cbb69212f941b22e5b6c3"}
Sep 10 23:38:13.682105 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.682075    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5564ccdb75-lnc69"]
Sep 10 23:38:13.685909 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.685889    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.688342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.688313    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 23:38:13.688452 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.688317    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 23:38:13.688528 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.688464    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-swqp2\""
Sep 10 23:38:13.688786 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.688771    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 23:38:13.689361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.689342    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 23:38:13.689460 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.689391    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 23:38:13.689460 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.689400    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-urblbsg3jts5\""
Sep 10 23:38:13.699287 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.699266    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5564ccdb75-lnc69"]
Sep 10 23:38:13.860850 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.860820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2q4j\" (UniqueName: \"kubernetes.io/projected/a1ecded2-81d8-436a-9a7a-0539697ada8b-kube-api-access-j2q4j\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.860882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.860914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-grpc-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.860983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.861026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ecded2-81d8-436a-9a7a-0539697ada8b-metrics-client-ca\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.861058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.861094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.861521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.861154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.961977 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.961955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ecded2-81d8-436a-9a7a-0539697ada8b-metrics-client-ca\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962056 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.961985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962056 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2q4j\" (UniqueName: \"kubernetes.io/projected/a1ecded2-81d8-436a-9a7a-0539697ada8b-kube-api-access-j2q4j\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962287 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-grpc-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962287 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.962759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.962735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ecded2-81d8-436a-9a7a-0539697ada8b-metrics-client-ca\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.965927 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.965876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.965927 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.965916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.966181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.966037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.966403 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.966381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.966587 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.966566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-grpc-tls\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.967336 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.967315    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a1ecded2-81d8-436a-9a7a-0539697ada8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.972647 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.972607    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2q4j\" (UniqueName: \"kubernetes.io/projected/a1ecded2-81d8-436a-9a7a-0539697ada8b-kube-api-access-j2q4j\") pod \"thanos-querier-5564ccdb75-lnc69\" (UID: \"a1ecded2-81d8-436a-9a7a-0539697ada8b\") " pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:13.998415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:13.998393    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:14.146062 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.146017    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:38:14.150409 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:14.150375    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c71a90_1b02_4dda_bbad_151de0610645.slice/crio-7754f46eb6a46aa80beb841213d633e9f5ef4f47d2cd7bb639efc4cf124c438b WatchSource:0}: Error finding container 7754f46eb6a46aa80beb841213d633e9f5ef4f47d2cd7bb639efc4cf124c438b: Status 404 returned error can't find the container with id 7754f46eb6a46aa80beb841213d633e9f5ef4f47d2cd7bb639efc4cf124c438b
Sep 10 23:38:14.321962 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.317705    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5564ccdb75-lnc69"]
Sep 10 23:38:14.325304 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:14.325275    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ecded2_81d8_436a_9a7a_0539697ada8b.slice/crio-586fac2f84b32a03012d6ca1509668f25880a97ae6ae3155ede22dc8c1888578 WatchSource:0}: Error finding container 586fac2f84b32a03012d6ca1509668f25880a97ae6ae3155ede22dc8c1888578: Status 404 returned error can't find the container with id 586fac2f84b32a03012d6ca1509668f25880a97ae6ae3155ede22dc8c1888578
Sep 10 23:38:14.636163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.636121    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"7754f46eb6a46aa80beb841213d633e9f5ef4f47d2cd7bb639efc4cf124c438b"}
Sep 10 23:38:14.640204 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.640176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-264xv" event={"ID":"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb","Type":"ContainerStarted","Data":"fe31e1b711488d8a59ebf3f2018e0dac722a87c805d717b4abed51fdc072cc3c"}
Sep 10 23:38:14.640395 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.640230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-264xv" event={"ID":"a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb","Type":"ContainerStarted","Data":"5f115267f9bd556501c439494978fef4290ef73a8524160f09e5c50c5a6da6fb"}
Sep 10 23:38:14.642247 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.642163    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"586fac2f84b32a03012d6ca1509668f25880a97ae6ae3155ede22dc8c1888578"}
Sep 10 23:38:14.646438 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.646411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9" event={"ID":"6c1df7c8-4c23-45d9-b873-865cce8c4bfa","Type":"ContainerStarted","Data":"4e8482bff08b0fb04aaf500ee6d3872306d21bb38866f17c1a81721d7d3a0400"}
Sep 10 23:38:14.646572 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.646548    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9" event={"ID":"6c1df7c8-4c23-45d9-b873-865cce8c4bfa","Type":"ContainerStarted","Data":"d7822f3a3454063681879239ad3a4919a5f47b0a706caa2b0709d115e2bd04c5"}
Sep 10 23:38:14.646641 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.646581    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9" event={"ID":"6c1df7c8-4c23-45d9-b873-865cce8c4bfa","Type":"ContainerStarted","Data":"ffdd534d587066cfd01a3f45faff9b3988421c623c4187252282cde2fadd811e"}
Sep 10 23:38:14.659844 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:14.659750    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-264xv" podStartSLOduration=2.705544835 podStartE2EDuration="3.659732796s" totalImagesPullingTime="884.935063ms" totalInitContainerRuntime="69.252898ms" isStatefulPod=true podCreationTimestamp="2026-09-10 23:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:14.659712755 +0000 UTC m=+84.925709626" watchObservedRunningTime="2026-09-10 23:38:14.659732796 +0000 UTC m=+84.925729668"
Sep 10 23:38:15.653877 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.653799    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828" exitCode=0
Sep 10 23:38:15.654545 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.654512    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"}
Sep 10 23:38:15.685102 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.684715    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-pv7d9" podStartSLOduration=3.576990563 podStartE2EDuration="4.684692809s" totalImagesPullingTime="1.107702246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:14.680345255 +0000 UTC m=+84.946342156" watchObservedRunningTime="2026-09-10 23:38:15.684692809 +0000 UTC m=+85.950689679"
Sep 10 23:38:15.866918 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.866882    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-775fb948db-9mzpc"]
Sep 10 23:38:15.870736 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.870710    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.873982 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.873951    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 23:38:15.874121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.873982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 23:38:15.874121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.874092    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 23:38:15.874471 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.874449    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 23:38:15.874745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.874728    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d7jq4m7ka1tqe\""
Sep 10 23:38:15.875087 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.874930    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nfqrw\""
Sep 10 23:38:15.887353 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.887294    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-775fb948db-9mzpc"]
Sep 10 23:38:15.982282 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdjs\" (UniqueName: \"kubernetes.io/projected/4b6d0514-a6cb-4515-b240-32ee0281e346-kube-api-access-7hdjs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982282 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-client-certs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-client-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-metrics-server-audit-profiles\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-tls\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:15.982735 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:15.982530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b6d0514-a6cb-4515-b240-32ee0281e346-audit-log\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083594 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b6d0514-a6cb-4515-b240-32ee0281e346-audit-log\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083750 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdjs\" (UniqueName: \"kubernetes.io/projected/4b6d0514-a6cb-4515-b240-32ee0281e346-kube-api-access-7hdjs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083750 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-client-certs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083750 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083750 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-client-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-metrics-server-audit-profiles\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.083963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.083824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-tls\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.084157 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.084126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b6d0514-a6cb-4515-b240-32ee0281e346-audit-log\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.084615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.084527    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.085173 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.085120    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b6d0514-a6cb-4515-b240-32ee0281e346-metrics-server-audit-profiles\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.086823 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.086801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-tls\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.086910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.086880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-secret-metrics-server-client-certs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.087258 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.087235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b6d0514-a6cb-4515-b240-32ee0281e346-client-ca-bundle\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.092545 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.092475    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdjs\" (UniqueName: \"kubernetes.io/projected/4b6d0514-a6cb-4515-b240-32ee0281e346-kube-api-access-7hdjs\") pod \"metrics-server-775fb948db-9mzpc\" (UID: \"4b6d0514-a6cb-4515-b240-32ee0281e346\") " pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.185319 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.185283    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:16.346998 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.346949    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"]
Sep 10 23:38:16.350313 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.350291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.352866 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.352674    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 23:38:16.352984 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.352968    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jrt9c\""
Sep 10 23:38:16.358534 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.358508    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"]
Sep 10 23:38:16.488013 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.487974    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4be27036-407a-4408-9f7d-a062184b68d2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.488133 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.488032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrn72\" (UniqueName: \"kubernetes.io/projected/4be27036-407a-4408-9f7d-a062184b68d2-kube-api-access-mrn72\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.495758 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.495731    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-775fb948db-9mzpc"]
Sep 10 23:38:16.499683 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:16.499655    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6d0514_a6cb_4515_b240_32ee0281e346.slice/crio-f0c9105d5c168d664525930ff09ef930c5a20bd9c2eeede469f931f1cedaceea WatchSource:0}: Error finding container f0c9105d5c168d664525930ff09ef930c5a20bd9c2eeede469f931f1cedaceea: Status 404 returned error can't find the container with id f0c9105d5c168d664525930ff09ef930c5a20bd9c2eeede469f931f1cedaceea
Sep 10 23:38:16.588765 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.588734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4be27036-407a-4408-9f7d-a062184b68d2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.588982 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.588784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrn72\" (UniqueName: \"kubernetes.io/projected/4be27036-407a-4408-9f7d-a062184b68d2-kube-api-access-mrn72\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.592066 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.592039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4be27036-407a-4408-9f7d-a062184b68d2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.596979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.596933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrn72\" (UniqueName: \"kubernetes.io/projected/4be27036-407a-4408-9f7d-a062184b68d2-kube-api-access-mrn72\") pod \"monitoring-plugin-5d85778b9d-f47t8\" (UID: \"4be27036-407a-4408-9f7d-a062184b68d2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.663789 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.663614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"0258032c89840845e6da140c2625dd01d7e839b5e8558c671e852b214fc68471"}
Sep 10 23:38:16.663789 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.663697    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"1f97e2582e81bfbfcd1dd7ce3a05af70257ac095b7ea8f216dfa39f349dc0067"}
Sep 10 23:38:16.663789 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.663742    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"131390d8ae18e650786ed91921826a9621078821b612b8bb03601d5a3a4fbe2b"}
Sep 10 23:38:16.665100 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.665077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:16.665311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.665291    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc" event={"ID":"4b6d0514-a6cb-4515-b240-32ee0281e346","Type":"ContainerStarted","Data":"f0c9105d5c168d664525930ff09ef930c5a20bd9c2eeede469f931f1cedaceea"}
Sep 10 23:38:16.755389 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.755040    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"]
Sep 10 23:38:16.760290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.760264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.762757    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.762851    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.762765    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.763151    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d7fn8\""
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.763353    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 23:38:16.763615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.763511    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 23:38:16.768525 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.768060    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 23:38:16.770009 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.769942    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"]
Sep 10 23:38:16.892871 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.892836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.892888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-serving-certs-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.892992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.893020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t84h\" (UniqueName: \"kubernetes.io/projected/22a78ef0-5b57-419a-998d-e19f47be5e71-kube-api-access-6t84h\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.893039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-federate-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893358 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.893125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-metrics-client-ca\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893358 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.893162    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.893358 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.893201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.948104 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.947995    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"]
Sep 10 23:38:16.994021 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.993978    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-metrics-client-ca\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-serving-certs-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994400 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994193    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994400 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t84h\" (UniqueName: \"kubernetes.io/projected/22a78ef0-5b57-419a-998d-e19f47be5e71-kube-api-access-6t84h\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.994400 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-federate-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.995311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.994970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-metrics-client-ca\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.995311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.995032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.995311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.995177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a78ef0-5b57-419a-998d-e19f47be5e71-serving-certs-ca-bundle\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.997912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.997868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-federate-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.997912 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.997882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-telemeter-client-tls\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:16.998257 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:16.998239    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:17.000401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.000360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22a78ef0-5b57-419a-998d-e19f47be5e71-secret-telemeter-client\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:17.003913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.003868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t84h\" (UniqueName: \"kubernetes.io/projected/22a78ef0-5b57-419a-998d-e19f47be5e71-kube-api-access-6t84h\") pod \"telemeter-client-9755dd4bf-7hp5z\" (UID: \"22a78ef0-5b57-419a-998d-e19f47be5e71\") " pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:17.074600 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.074294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"
Sep 10 23:38:17.448099 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:17.447718    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be27036_407a_4408_9f7d_a062184b68d2.slice/crio-cb40edda43ff984a25b6bec0de7ad780bdd0d93e69024dccceab3adf2bdc529e WatchSource:0}: Error finding container cb40edda43ff984a25b6bec0de7ad780bdd0d93e69024dccceab3adf2bdc529e: Status 404 returned error can't find the container with id cb40edda43ff984a25b6bec0de7ad780bdd0d93e69024dccceab3adf2bdc529e
Sep 10 23:38:17.673998 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.671993    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8" event={"ID":"4be27036-407a-4408-9f7d-a062184b68d2","Type":"ContainerStarted","Data":"cb40edda43ff984a25b6bec0de7ad780bdd0d93e69024dccceab3adf2bdc529e"}
Sep 10 23:38:17.947222 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.947183    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:38:17.952942 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.952910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:17.955327 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.955295    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v9fcf\""
Sep 10 23:38:17.955568 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.955551    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.955987    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.956216    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.956402    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.956647    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.956819    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 23:38:17.957410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.956982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 23:38:17.959125 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.959101    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.959536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.959754    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.960015    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.960293    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-978sliidhrctj\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.960535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 23:38:17.960873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.960752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 23:38:17.962388 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.962363    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 23:38:17.969638 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:17.968450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:38:18.106517 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.106742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.106742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.106742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.106742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.106742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6pk\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106931    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106960    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.106988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.107026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.107922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.108096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.108368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.108152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209426    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6pk\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.209777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209809    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.209918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210673 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.210652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.210760 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.210738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.216261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.215905    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.216261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.216253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.217268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.216862    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.217268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.217268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.217432 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.219027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.219027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.219027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.217910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.219027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.218587    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.219754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.219711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.221216 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.220921    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.221216 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.220945    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.221216 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.221193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.223614 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.223404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.225857 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.225613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.226997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.226957    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.230473 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.230450    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6pk\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk\") pod \"prometheus-k8s-0\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.271003 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.270804    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:18.553714 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.553388    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9755dd4bf-7hp5z"]
Sep 10 23:38:18.557937 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:18.557894    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a78ef0_5b57_419a_998d_e19f47be5e71.slice/crio-256bd82320419afbf99d83b201acad652e5577b38b8a1091a4f79eeaedceaed0 WatchSource:0}: Error finding container 256bd82320419afbf99d83b201acad652e5577b38b8a1091a4f79eeaedceaed0: Status 404 returned error can't find the container with id 256bd82320419afbf99d83b201acad652e5577b38b8a1091a4f79eeaedceaed0
Sep 10 23:38:18.636638 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.636599    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:38:18.641290 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:38:18.641141    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c26bba8_37f8_4f01_971d_0b356b6fef53.slice/crio-bae7648d354dfc13a780733f2904e74fd09a48489a44531715c54e2c428deb15 WatchSource:0}: Error finding container bae7648d354dfc13a780733f2904e74fd09a48489a44531715c54e2c428deb15: Status 404 returned error can't find the container with id bae7648d354dfc13a780733f2904e74fd09a48489a44531715c54e2c428deb15
Sep 10 23:38:18.682668 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.682576    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z" event={"ID":"22a78ef0-5b57-419a-998d-e19f47be5e71","Type":"ContainerStarted","Data":"256bd82320419afbf99d83b201acad652e5577b38b8a1091a4f79eeaedceaed0"}
Sep 10 23:38:18.688825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.688638    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"}
Sep 10 23:38:18.688825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.688707    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"}
Sep 10 23:38:18.688825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.688754    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"}
Sep 10 23:38:18.688825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.688792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"}
Sep 10 23:38:18.694281 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.693889    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"8b942aa2983729923ced1feb19e781864ef3256d09f009806cead4243ad26b99"}
Sep 10 23:38:18.694281 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.693964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"3245a072fd655a5e5dd80fd69e2d75a22830be72f9eae38db66c88de584f1889"}
Sep 10 23:38:18.694281 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.694006    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" event={"ID":"a1ecded2-81d8-436a-9a7a-0539697ada8b","Type":"ContainerStarted","Data":"9f3826f22ded7bea63717793607de9d35caad9927b89ed9cfd2e0a1ecc12a395"}
Sep 10 23:38:18.695951 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.694786    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:18.698551 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.697939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc" event={"ID":"4b6d0514-a6cb-4515-b240-32ee0281e346","Type":"ContainerStarted","Data":"fb4d6b605ebf2cfdeff51e534c35d459486b419f1855509874a096f48cb8300a"}
Sep 10 23:38:18.700387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.700311    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"bae7648d354dfc13a780733f2904e74fd09a48489a44531715c54e2c428deb15"}
Sep 10 23:38:18.717550 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:18.717000    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69" podStartSLOduration=2.035713162 podStartE2EDuration="5.716979528s" totalImagesPullingTime="3.681266366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:18.713757465 +0000 UTC m=+88.979754361" watchObservedRunningTime="2026-09-10 23:38:18.716979528 +0000 UTC m=+88.982976400"
Sep 10 23:38:19.712233 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.712192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"}
Sep 10 23:38:19.712657 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.712319    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerStarted","Data":"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"}
Sep 10 23:38:19.714632 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.714596    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499" exitCode=0
Sep 10 23:38:19.714780 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.714681    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"}
Sep 10 23:38:19.717239 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.717190    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8" event={"ID":"4be27036-407a-4408-9f7d-a062184b68d2","Type":"ContainerStarted","Data":"535406644b6434aaed92c6dce7400f88e539a58a841655b82f40de4bd0db5818"}
Sep 10 23:38:19.739290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.739222    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.955429455 podStartE2EDuration="7.739204426s" totalImagesPullingTime="3.7641662s" totalInitContainerRuntime="19.608771ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:19.737725307 +0000 UTC m=+90.003722206" watchObservedRunningTime="2026-09-10 23:38:19.739204426 +0000 UTC m=+90.005201298"
Sep 10 23:38:19.739456 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.739430    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc" podStartSLOduration=2.920426044 podStartE2EDuration="4.739423113s" totalImagesPullingTime="1.818997069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:18.732666028 +0000 UTC m=+88.998662899" watchObservedRunningTime="2026-09-10 23:38:19.739423113 +0000 UTC m=+90.005420001"
Sep 10 23:38:19.779206 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:19.779085    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8" podStartSLOduration=1.710736567 podStartE2EDuration="3.779069376s" totalImagesPullingTime="2.068332809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:19.778330283 +0000 UTC m=+90.044327174" watchObservedRunningTime="2026-09-10 23:38:19.779069376 +0000 UTC m=+90.045066250"
Sep 10 23:38:20.730182 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.730100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z" event={"ID":"22a78ef0-5b57-419a-998d-e19f47be5e71","Type":"ContainerStarted","Data":"fafc299d585615fbc5149ef9b29acbb79f1e6563c452acb584e81dd24775e30d"}
Sep 10 23:38:20.730630 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.730203    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z" event={"ID":"22a78ef0-5b57-419a-998d-e19f47be5e71","Type":"ContainerStarted","Data":"e70c90431633e9c50061d300bc620c150b5a30d0a063f88c3310a2d7ae9337f6"}
Sep 10 23:38:20.730630 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.730233    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z" event={"ID":"22a78ef0-5b57-419a-998d-e19f47be5e71","Type":"ContainerStarted","Data":"cbe3811fbae51c20528883a9aa945145bfc76efee6f503a32aa7a460412073bf"}
Sep 10 23:38:20.730630 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.730373    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:20.736769 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.736743    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f47t8"
Sep 10 23:38:20.754150 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:20.754104    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9755dd4bf-7hp5z" podStartSLOduration=2.8694709400000002 podStartE2EDuration="4.754084789s" totalImagesPullingTime="1.884613849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:20.752316153 +0000 UTC m=+91.018313048" watchObservedRunningTime="2026-09-10 23:38:20.754084789 +0000 UTC m=+91.020081661"
Sep 10 23:38:23.750978 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:23.750793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"}
Sep 10 23:38:23.750978 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:23.750875    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"}
Sep 10 23:38:24.725356 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.725324    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5564ccdb75-lnc69"
Sep 10 23:38:24.762875 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.762842    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"}
Sep 10 23:38:24.763328 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.762919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"}
Sep 10 23:38:24.763328 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.763012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"}
Sep 10 23:38:24.763328 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.763057    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerStarted","Data":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"}
Sep 10 23:38:24.790252 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:24.790198    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.902693579 podStartE2EDuration="7.790176986s" totalImagesPullingTime="3.858408455s" totalInitContainerRuntime="29.074952ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:24.787913524 +0000 UTC m=+95.053910440" watchObservedRunningTime="2026-09-10 23:38:24.790176986 +0000 UTC m=+95.056173858"
Sep 10 23:38:28.271855 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:28.271816    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:38:32.568675 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.568650    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s2mbj"
Sep 10 23:38:32.811901 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.811840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-8clcw" event={"ID":"c666dc5c-ed83-4620-a4a9-954acdf30026","Type":"ContainerStarted","Data":"38de1cd25faa53a9ff0f33d75a0489ed412da43283c709879da683c95e3b9a27"}
Sep 10 23:38:32.812092 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.812071    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:32.813480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.813455    2575 patch_prober.go:28] interesting pod/downloads-76f99584f4-8clcw container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.18:8080/\": dial tcp 10.134.0.18:8080: connect: connection refused" start-of-body=
Sep 10 23:38:32.813630 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.813558    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-8clcw" podUID="c666dc5c-ed83-4620-a4a9-954acdf30026" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.18:8080/\": dial tcp 10.134.0.18:8080: connect: connection refused"
Sep 10 23:38:32.828882 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:32.828792    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-8clcw" podStartSLOduration=1.6361179369999999 podStartE2EDuration="20.828778939s" totalImagesPullingTime="19.192661002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:32.826646314 +0000 UTC m=+103.092643211" watchObservedRunningTime="2026-09-10 23:38:32.828778939 +0000 UTC m=+103.094775810"
Sep 10 23:38:33.832246 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:33.832210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-8clcw"
Sep 10 23:38:36.185693 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:36.185657    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:36.186250 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:36.185730    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:37.839079 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:37.839046    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8802c81-0116-4bb9-abbe-f2da5c7cb762" containerID="1fb4281a099ec3d6f9bc5edb017537494edd9c06fb53cb5c5f2845402ad45e5a" exitCode=0
Sep 10 23:38:37.839572 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:37.839118    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84" event={"ID":"a8802c81-0116-4bb9-abbe-f2da5c7cb762","Type":"ContainerDied","Data":"1fb4281a099ec3d6f9bc5edb017537494edd9c06fb53cb5c5f2845402ad45e5a"}
Sep 10 23:38:37.839572 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:37.839481    2575 scope.go:117] "RemoveContainer" containerID="1fb4281a099ec3d6f9bc5edb017537494edd9c06fb53cb5c5f2845402ad45e5a"
Sep 10 23:38:38.847247 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:38.847208    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b8v84" event={"ID":"a8802c81-0116-4bb9-abbe-f2da5c7cb762","Type":"ContainerStarted","Data":"422b3458968fd12eff403cf488956064fcd6a9f1cf5ee183d33d178b6f6b6e2e"}
Sep 10 23:38:52.197815 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:52.197786    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55546df84-vmltb_8c541dc9-c33b-483a-8502-7dcdff60c7fe/router/0.log"
Sep 10 23:38:52.216815 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:52.216790    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf4f5_d53c74b5-5763-455e-8e99-a6446c62421b/serve-healthcheck-canary/0.log"
Sep 10 23:38:56.191434 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:56.191399    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:38:56.195188 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:38:56.195164    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-775fb948db-9mzpc"
Sep 10 23:39:01.949006 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:01.948973    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba1d9df4-9515-4ce2-95f8-323bdd3c2b05" containerID="7571ae2a416dff9dd12cecf04605f8c33cc9f99f01a9751ca65854e2c1f64a99" exitCode=0
Sep 10 23:39:01.949361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:01.949037    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd" event={"ID":"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05","Type":"ContainerDied","Data":"7571ae2a416dff9dd12cecf04605f8c33cc9f99f01a9751ca65854e2c1f64a99"}
Sep 10 23:39:01.949361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:01.949346    2575 scope.go:117] "RemoveContainer" containerID="7571ae2a416dff9dd12cecf04605f8c33cc9f99f01a9751ca65854e2c1f64a99"
Sep 10 23:39:02.955790 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:02.955747    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7xpbd" event={"ID":"ba1d9df4-9515-4ce2-95f8-323bdd3c2b05","Type":"ContainerStarted","Data":"87c1feaaf0c76ce7d2207ce2a035093a51bef1af28043f43cf7f3811d09ba867"}
Sep 10 23:39:18.272077 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:18.272044    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:18.288910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:18.288884    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:19.045821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:19.045795    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:32.031941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.031904    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032314    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="alertmanager" containerID="cri-o://b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac" gracePeriod=120
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032330    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy" containerID="cri-o://671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8" gracePeriod=120
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032351    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="prom-label-proxy" containerID="cri-o://fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b" gracePeriod=120
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032381    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-metric" containerID="cri-o://6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d" gracePeriod=120
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032381    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-web" containerID="cri-o://d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c" gracePeriod=120
Sep 10 23:39:32.032552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:32.032434    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="config-reloader" containerID="cri-o://bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f" gracePeriod=120
Sep 10 23:39:33.089907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.089876    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b" exitCode=0
Sep 10 23:39:33.089907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.089897    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8" exitCode=0
Sep 10 23:39:33.089907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.089904    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f" exitCode=0
Sep 10 23:39:33.089907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.089910    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac" exitCode=0
Sep 10 23:39:33.090364 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.089944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"}
Sep 10 23:39:33.090364 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.090006    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"}
Sep 10 23:39:33.090364 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.090034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"}
Sep 10 23:39:33.090364 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.090057    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"}
Sep 10 23:39:33.270118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.270094    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:33.366167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366091    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366126    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366144    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366167    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366185    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366312    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366380    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366426    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366463    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366524    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcs6q\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366556    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366553    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:33.366742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366585    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366612    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle\") pod \"66c71a90-1b02-4dda-bbad-151de0610645\" (UID: \"66c71a90-1b02-4dda-bbad-151de0610645\") "
Sep 10 23:39:33.366964 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.366898    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-metrics-client-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.367466 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.367228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:33.368060 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.367998    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:39:33.369639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.369605    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out" (OuterVolumeSpecName: "config-out") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:39:33.370172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:39:33.370172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370134    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.370305 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370212    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.370380 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370362    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.370883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.370883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.370854    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume" (OuterVolumeSpecName: "config-volume") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.371628 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.371605    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q" (OuterVolumeSpecName: "kube-api-access-dcs6q") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "kube-api-access-dcs6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:39:33.374796 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.374702    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.381665 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.381644    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config" (OuterVolumeSpecName: "web-config") pod "66c71a90-1b02-4dda-bbad-151de0610645" (UID: "66c71a90-1b02-4dda-bbad-151de0610645"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:33.467718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467693    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-config-out\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467716    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-tls-assets\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467729    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-cluster-tls-config\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467742    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-main-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467755    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-web-config\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467767    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467779    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-main-db\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467792    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467805    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcs6q\" (UniqueName: \"kubernetes.io/projected/66c71a90-1b02-4dda-bbad-151de0610645-kube-api-access-dcs6q\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467817    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467830    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/66c71a90-1b02-4dda-bbad-151de0610645-config-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:33.467883 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:33.467844    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c71a90-1b02-4dda-bbad-151de0610645-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:34.098191 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098159    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d" exitCode=0
Sep 10 23:39:34.098191 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098187    2575 generic.go:356] "Generic (PLEG): container finished" podID="66c71a90-1b02-4dda-bbad-151de0610645" containerID="d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c" exitCode=0
Sep 10 23:39:34.098629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098235    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.098629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"}
Sep 10 23:39:34.098629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"}
Sep 10 23:39:34.098629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098333    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"66c71a90-1b02-4dda-bbad-151de0610645","Type":"ContainerDied","Data":"7754f46eb6a46aa80beb841213d633e9f5ef4f47d2cd7bb639efc4cf124c438b"}
Sep 10 23:39:34.098629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.098366    2575 scope.go:117] "RemoveContainer" containerID="fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"
Sep 10 23:39:34.108886 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.108870    2575 scope.go:117] "RemoveContainer" containerID="6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"
Sep 10 23:39:34.116091 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.116074    2575 scope.go:117] "RemoveContainer" containerID="671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"
Sep 10 23:39:34.121248 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.121223    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:34.124342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.124321    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:34.124820 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.124804    2575 scope.go:117] "RemoveContainer" containerID="d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"
Sep 10 23:39:34.131615 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.131597    2575 scope.go:117] "RemoveContainer" containerID="bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"
Sep 10 23:39:34.138711 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.138698    2575 scope.go:117] "RemoveContainer" containerID="b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"
Sep 10 23:39:34.146692 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.146476    2575 scope.go:117] "RemoveContainer" containerID="ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"
Sep 10 23:39:34.148940 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.148918    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:34.149261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149249    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="config-reloader"
Sep 10 23:39:34.149301 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149264    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="config-reloader"
Sep 10 23:39:34.149301 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149277    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-web"
Sep 10 23:39:34.149301 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149285    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-web"
Sep 10 23:39:34.149301 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149298    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="alertmanager"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149306    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="alertmanager"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149311    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="prom-label-proxy"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149316    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="prom-label-proxy"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149328    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149333    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149339    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="init-config-reloader"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149344    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="init-config-reloader"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149351    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-metric"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-metric"
Sep 10 23:39:34.149415 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149414    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy"
Sep 10 23:39:34.149729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149425    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="prom-label-proxy"
Sep 10 23:39:34.149729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149432    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="config-reloader"
Sep 10 23:39:34.149729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149444    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="alertmanager"
Sep 10 23:39:34.149729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149450    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-web"
Sep 10 23:39:34.149729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.149455    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c71a90-1b02-4dda-bbad-151de0610645" containerName="kube-rbac-proxy-metric"
Sep 10 23:39:34.156367 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.155757    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.160886 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.160864    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 23:39:34.161171 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2lqsh\""
Sep 10 23:39:34.161245 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 23:39:34.161245 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161201    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 23:39:34.161355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161244    2575 scope.go:117] "RemoveContainer" containerID="fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"
Sep 10 23:39:34.161355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 23:39:34.161649 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161629    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 23:39:34.161737 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.161643    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b\": container with ID starting with fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b not found: ID does not exist" containerID="fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"
Sep 10 23:39:34.161737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161669    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 23:39:34.161737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161674    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"} err="failed to get container status \"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b\": rpc error: code = NotFound desc = could not find container \"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b\": container with ID starting with fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b not found: ID does not exist"
Sep 10 23:39:34.161737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161715    2575 scope.go:117] "RemoveContainer" containerID="6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"
Sep 10 23:39:34.161988 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.161823    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 23:39:34.162052 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162039    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 23:39:34.162052 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.162042    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d\": container with ID starting with 6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d not found: ID does not exist" containerID="6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"
Sep 10 23:39:34.162127 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162064    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"} err="failed to get container status \"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d\": rpc error: code = NotFound desc = could not find container \"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d\": container with ID starting with 6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d not found: ID does not exist"
Sep 10 23:39:34.162127 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162079    2575 scope.go:117] "RemoveContainer" containerID="671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"
Sep 10 23:39:34.162424 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.162398    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8\": container with ID starting with 671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8 not found: ID does not exist" containerID="671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"
Sep 10 23:39:34.162546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162429    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"} err="failed to get container status \"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8\": rpc error: code = NotFound desc = could not find container \"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8\": container with ID starting with 671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8 not found: ID does not exist"
Sep 10 23:39:34.162546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162443    2575 scope.go:117] "RemoveContainer" containerID="d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"
Sep 10 23:39:34.162745 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.162726    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c\": container with ID starting with d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c not found: ID does not exist" containerID="d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"
Sep 10 23:39:34.162815 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162750    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"} err="failed to get container status \"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c\": rpc error: code = NotFound desc = could not find container \"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c\": container with ID starting with d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c not found: ID does not exist"
Sep 10 23:39:34.162815 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.162764    2575 scope.go:117] "RemoveContainer" containerID="bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"
Sep 10 23:39:34.163021 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.163004    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f\": container with ID starting with bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f not found: ID does not exist" containerID="bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"
Sep 10 23:39:34.163086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163029    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"} err="failed to get container status \"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f\": rpc error: code = NotFound desc = could not find container \"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f\": container with ID starting with bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f not found: ID does not exist"
Sep 10 23:39:34.163086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163049    2575 scope.go:117] "RemoveContainer" containerID="b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"
Sep 10 23:39:34.163343 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.163324    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac\": container with ID starting with b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac not found: ID does not exist" containerID="b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"
Sep 10 23:39:34.163421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163347    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"} err="failed to get container status \"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac\": rpc error: code = NotFound desc = could not find container \"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac\": container with ID starting with b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac not found: ID does not exist"
Sep 10 23:39:34.163421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163362    2575 scope.go:117] "RemoveContainer" containerID="ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"
Sep 10 23:39:34.163683 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:34.163667    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828\": container with ID starting with ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828 not found: ID does not exist" containerID="ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"
Sep 10 23:39:34.163747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163687    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"} err="failed to get container status \"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828\": rpc error: code = NotFound desc = could not find container \"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828\": container with ID starting with ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828 not found: ID does not exist"
Sep 10 23:39:34.163747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163708    2575 scope.go:117] "RemoveContainer" containerID="fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"
Sep 10 23:39:34.163977 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163956    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b"} err="failed to get container status \"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b\": rpc error: code = NotFound desc = could not find container \"fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b\": container with ID starting with fedd8f341c575e3505ec5115ff72b26eac53a61fcdfc714b626ffec17004606b not found: ID does not exist"
Sep 10 23:39:34.163977 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.163978    2575 scope.go:117] "RemoveContainer" containerID="6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"
Sep 10 23:39:34.164482 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.164459    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d"} err="failed to get container status \"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d\": rpc error: code = NotFound desc = could not find container \"6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d\": container with ID starting with 6fe9463612e194d11ebb7a29b4d6b240be4263b1b696f10c9126ae6bc353423d not found: ID does not exist"
Sep 10 23:39:34.164482 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.164480    2575 scope.go:117] "RemoveContainer" containerID="671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"
Sep 10 23:39:34.164852 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.164829    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8"} err="failed to get container status \"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8\": rpc error: code = NotFound desc = could not find container \"671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8\": container with ID starting with 671d055a91a3e0cdb2c99f784755adf012dce97966380a73f70a6152c325fed8 not found: ID does not exist"
Sep 10 23:39:34.164852 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.164853    2575 scope.go:117] "RemoveContainer" containerID="d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"
Sep 10 23:39:34.165180 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165144    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c"} err="failed to get container status \"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c\": rpc error: code = NotFound desc = could not find container \"d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c\": container with ID starting with d434a3f9b582aa03a34c0543cc1b83f034aed265e1d5c2f8f6601e285bd7fa4c not found: ID does not exist"
Sep 10 23:39:34.165180 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165169    2575 scope.go:117] "RemoveContainer" containerID="bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"
Sep 10 23:39:34.165463 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165445    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f"} err="failed to get container status \"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f\": rpc error: code = NotFound desc = could not find container \"bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f\": container with ID starting with bafea63a7f76d146aa51853259cf8d79fdab55ed94492c253d3fd03448e8996f not found: ID does not exist"
Sep 10 23:39:34.165550 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165472    2575 scope.go:117] "RemoveContainer" containerID="b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"
Sep 10 23:39:34.165736 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165719    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac"} err="failed to get container status \"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac\": rpc error: code = NotFound desc = could not find container \"b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac\": container with ID starting with b60f84b3420d16a0965eadc15e0d966c42a4c36d76c63436781fc552bac43bac not found: ID does not exist"
Sep 10 23:39:34.165792 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165737    2575 scope.go:117] "RemoveContainer" containerID="ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"
Sep 10 23:39:34.165976 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.165960    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828"} err="failed to get container status \"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828\": rpc error: code = NotFound desc = could not find container \"ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828\": container with ID starting with ffab0c40ef1013cf5899e64918985bde36e21f58abaf57dc9c30f95a8dbd6828 not found: ID does not exist"
Sep 10 23:39:34.172152 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.172129    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 23:39:34.173064 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.173046    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:34.227310 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.227283    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66c71a90-1b02-4dda-bbad-151de0610645" path="/var/lib/kubelet/pods/66c71a90-1b02-4dda-bbad-151de0610645/volumes"
Sep 10 23:39:34.273636 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-config-out\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-config-volume\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-web-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273940 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv8g5\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-kube-api-access-cv8g5\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273940 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.273940 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.274054 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.274054 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.273999    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.274054 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.274020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.274054 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.274044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.274198 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.274066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375442    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375664    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-config-out\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-config-volume\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.375990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.375975    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-web-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.376304 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.376007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv8g5\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-kube-api-access-cv8g5\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.376372 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.376269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.376780 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.376720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/391756d2-42bb-4580-b22a-eba6b48132f2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.378950 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.378922    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.378950 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.378939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.379095 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.378975    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.379173 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.379152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.379228 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.379180    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/391756d2-42bb-4580-b22a-eba6b48132f2-config-out\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.379322 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.379271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.379545 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.379480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.380040 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.380019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-config-volume\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.381047 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.381032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/391756d2-42bb-4580-b22a-eba6b48132f2-web-config\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.385261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.385238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv8g5\" (UniqueName: \"kubernetes.io/projected/391756d2-42bb-4580-b22a-eba6b48132f2-kube-api-access-cv8g5\") pod \"alertmanager-main-0\" (UID: \"391756d2-42bb-4580-b22a-eba6b48132f2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.466796 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.466774    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:39:34.749358 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:34.749270    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:39:34.752163 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:39:34.752140    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391756d2_42bb_4580_b22a_eba6b48132f2.slice/crio-fbe1d0d1480a137bff73f4e8aa5be4cb943b9546558f52d4a5ebcacd9e6b06a7 WatchSource:0}: Error finding container fbe1d0d1480a137bff73f4e8aa5be4cb943b9546558f52d4a5ebcacd9e6b06a7: Status 404 returned error can't find the container with id fbe1d0d1480a137bff73f4e8aa5be4cb943b9546558f52d4a5ebcacd9e6b06a7
Sep 10 23:39:35.104357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:35.104324    2575 generic.go:356] "Generic (PLEG): container finished" podID="391756d2-42bb-4580-b22a-eba6b48132f2" containerID="a3c2594d4800da1572e9c1ace2dd743f6284a209184aa97911f711e6d6afa8e9" exitCode=0
Sep 10 23:39:35.104776 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:35.104416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerDied","Data":"a3c2594d4800da1572e9c1ace2dd743f6284a209184aa97911f711e6d6afa8e9"}
Sep 10 23:39:35.104776 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:35.104517    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"fbe1d0d1480a137bff73f4e8aa5be4cb943b9546558f52d4a5ebcacd9e6b06a7"}
Sep 10 23:39:36.114882 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.114853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"c3909336a94f2965b3c6da953ded87de17a555c8ef110b651f630af4acdcc709"}
Sep 10 23:39:36.115260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.114914    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"148eacc27753b5d1d0ef165191800ebf77191393fbf0cd82273b8a3c78506f72"}
Sep 10 23:39:36.115260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.114939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"0be6157639789e56d8be5a75d3964bbcb8183419438d5cf8a566da03570e0f68"}
Sep 10 23:39:36.115260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.114969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"e8f1bff09be634d4ddf5fd625a192e08a5f31d7cda6bd546c5509836f03349d0"}
Sep 10 23:39:36.115260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.114993    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"d680b3c082593f7c34f62ee601524fb412d3856e84eb0cc9e7cd42c4ef85fa86"}
Sep 10 23:39:36.115260 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.115016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"391756d2-42bb-4580-b22a-eba6b48132f2","Type":"ContainerStarted","Data":"0ee3ede151b64753642055f25f57dbe1f42ca5c1c38eb428656b4c241c9b323d"}
Sep 10 23:39:36.139041 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.138984    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.121655694 podStartE2EDuration="2.138966256s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.310562ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:36.138528506 +0000 UTC m=+166.404525377" watchObservedRunningTime="2026-09-10 23:39:36.138966256 +0000 UTC m=+166.404963128"
Sep 10 23:39:36.250311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250281    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:36.250688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250635    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="prometheus" containerID="cri-o://fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353" gracePeriod=600
Sep 10 23:39:36.250818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250737    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="config-reloader" containerID="cri-o://a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64" gracePeriod=600
Sep 10 23:39:36.252577 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250653    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy" containerID="cri-o://bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952" gracePeriod=600
Sep 10 23:39:36.252577 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250687    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-thanos" containerID="cri-o://15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04" gracePeriod=600
Sep 10 23:39:36.252577 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250694    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-web" containerID="cri-o://fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880" gracePeriod=600
Sep 10 23:39:36.253300 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.250661    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="thanos-sidecar" containerID="cri-o://e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5" gracePeriod=600
Sep 10 23:39:36.504700 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.504676    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:36.596426 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596398    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596445    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596468    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596515    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596545    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596572    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596600    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596631    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596686    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596806    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596830    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596857    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596886    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv6pk\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.596926 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.596973    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597004    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597030    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597028    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597064    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2c26bba8-37f8-4f01-971d-0b356b6fef53\" (UID: \"2c26bba8-37f8-4f01-971d-0b356b6fef53\") "
Sep 10 23:39:36.597361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597300    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.597661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597385    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.597661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597402    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.597661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.597417    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.599025 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.598646    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.599254 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.599228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.599854 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.599824    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.600071 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.600037    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:39:36.600400 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.600377    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:39:36.600746 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.600676    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config" (OuterVolumeSpecName: "config") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.600888 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.600859    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.601344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.601314    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.601429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.601413    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out" (OuterVolumeSpecName: "config-out") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:39:36.602015 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.601523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.602669 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.602633    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.602838 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.602807    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.603134 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.603115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.603229 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.603204    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk" (OuterVolumeSpecName: "kube-api-access-kv6pk") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "kube-api-access-kv6pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:39:36.603435 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.603410    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.603484 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.603449    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:39:36.613335 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.613316    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config" (OuterVolumeSpecName: "web-config") pod "2c26bba8-37f8-4f01-971d-0b356b6fef53" (UID: "2c26bba8-37f8-4f01-971d-0b356b6fef53"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697771    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-tls-assets\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697795    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697806    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv6pk\" (UniqueName: \"kubernetes.io/projected/2c26bba8-37f8-4f01-971d-0b356b6fef53-kube-api-access-kv6pk\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697815    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697824    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-metrics-client-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.697835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697833    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-db\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697842    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-configmap-metrics-client-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697851    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697859    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-kube-rbac-proxy\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697867    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-grpc-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697895    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-web-config\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697903    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697911    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c26bba8-37f8-4f01-971d-0b356b6fef53-config-out\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697919    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-config\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697926    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697934    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c26bba8-37f8-4f01-971d-0b356b6fef53-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:36.698082 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:36.697943    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c26bba8-37f8-4f01-971d-0b356b6fef53-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:39:37.122298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122269    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04" exitCode=0
Sep 10 23:39:37.122298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122291    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952" exitCode=0
Sep 10 23:39:37.122298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122299    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880" exitCode=0
Sep 10 23:39:37.122298 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122304    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5" exitCode=0
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122310    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64" exitCode=0
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122316    2575 generic.go:356] "Generic (PLEG): container finished" podID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353" exitCode=0
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122339    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122449    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122514    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122518    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122566    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122646    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"}
Sep 10 23:39:37.122865 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.122682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c26bba8-37f8-4f01-971d-0b356b6fef53","Type":"ContainerDied","Data":"bae7648d354dfc13a780733f2904e74fd09a48489a44531715c54e2c428deb15"}
Sep 10 23:39:37.133425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.133407    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.140618 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.140599    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.147627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.147609    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.149383 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.149360    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:37.153673 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.153655    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:37.155947 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.155926    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.162811 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.162795    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.170042 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.170026    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.176756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.176740    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.177052 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.177030    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.177093 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.177064    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.177093 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.177088    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.178003 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.177975    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.178138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178009    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.178138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178030    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.178336 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.178314    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.178388 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178345    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.178388 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178366    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.178679 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.178659    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.178753 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178686    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.178753 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178704    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.178971 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.178945    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.179012 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178969    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.179012 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.178987    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.179230 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.179215    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.179266 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179233    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.179266 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179245    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.179485 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:39:37.179469    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.179485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179507    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.179485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179524    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.179485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179529    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:37.179731 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179711    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.179772 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179732    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.179895 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179881    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="config-reloader"
Sep 10 23:39:37.179947 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179899    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="config-reloader"
Sep 10 23:39:37.179947 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179913    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-thanos"
Sep 10 23:39:37.179947 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179936    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-thanos"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179947    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="init-config-reloader"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179954    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="init-config-reloader"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179960    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-web"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179965    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-web"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179973    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="thanos-sidecar"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179978    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="thanos-sidecar"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179987    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="prometheus"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179992    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="prometheus"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179910    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180004    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.180034 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.179997    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180043    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180139    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-web"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180157    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180167    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="prometheus"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180174    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="thanos-sidecar"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180183    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="config-reloader"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180191    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" containerName="kube-rbac-proxy-thanos"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180285    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.180362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180299    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.180666 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180518    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.180666 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180534    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.180776 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180760    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.180814 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180778    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.180996 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180980    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.181042 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.180996    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.181185 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181168    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.181224 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181184    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.181399 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181379    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.181438 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181400    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.181634 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181619    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.181689 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181634    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.181886 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181859    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.181886 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.181880    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.182141 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182126    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.182188 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182142    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.182449 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182425    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.182449 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182446    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.182779 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182759    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.182834 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.182779    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.183021 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183001    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.183093 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183022    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.183318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183298    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.183370 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183320    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.183627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183599    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.183627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183627    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.183860 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183843    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.183909 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.183860    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.184060 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.184042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.184104 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.184089    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.184104 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.184100    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.184820 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.184412    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.184820 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.184445    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.185048 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185028    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.185124 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185050    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.185354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185336    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.185396 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185355    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.185645 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185619    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.185704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185648    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.185907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185887    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.185966 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.185908    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.186147 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186127    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.186214 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186146    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.186375 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186350    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.186375 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186375    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.186563 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 23:39:37.186563 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 23:39:37.186563 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186549    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-978sliidhrctj\""
Sep 10 23:39:37.186707 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186586    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 23:39:37.186707 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v9fcf\""
Sep 10 23:39:37.186913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186821    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 23:39:37.186913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186812    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.186913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186841    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.187062 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186923    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 23:39:37.187062 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.186994    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 23:39:37.187166 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187119    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.187166 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187136    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.187271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187181    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 23:39:37.187271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187239    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 23:39:37.187271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187263    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 23:39:37.187401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187311    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 23:39:37.187401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187334    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 23:39:37.187401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187242    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 23:39:37.187689 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187670    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.187689 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.187690    2575 scope.go:117] "RemoveContainer" containerID="15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"
Sep 10 23:39:37.188218 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.188190    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04"} err="failed to get container status \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": rpc error: code = NotFound desc = could not find container \"15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04\": container with ID starting with 15f06099bf42fa010e30f55ff7e8e3fa2729472e1b43c0539ce1f66ccd6cbc04 not found: ID does not exist"
Sep 10 23:39:37.188344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.188221    2575 scope.go:117] "RemoveContainer" containerID="bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"
Sep 10 23:39:37.188737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.188631    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952"} err="failed to get container status \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": rpc error: code = NotFound desc = could not find container \"bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952\": container with ID starting with bf692d5616656cb2bff1056f51e66098c281052f25ef7a043a48c34e35736952 not found: ID does not exist"
Sep 10 23:39:37.188737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.188658    2575 scope.go:117] "RemoveContainer" containerID="fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"
Sep 10 23:39:37.189406 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.189378    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880"} err="failed to get container status \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": rpc error: code = NotFound desc = could not find container \"fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880\": container with ID starting with fed1fae640c4a5df93a5d63a93d759110ba0b10650484bb28c0510ba52db1880 not found: ID does not exist"
Sep 10 23:39:37.189406 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.189404    2575 scope.go:117] "RemoveContainer" containerID="e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"
Sep 10 23:39:37.190521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.189808    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5"} err="failed to get container status \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": rpc error: code = NotFound desc = could not find container \"e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5\": container with ID starting with e1d6f9826bba5d7b8b6688fc5536e9ee41d521bb1da4adb0e48438cbb43c6da5 not found: ID does not exist"
Sep 10 23:39:37.190521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.189850    2575 scope.go:117] "RemoveContainer" containerID="a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"
Sep 10 23:39:37.190521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.190202    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64"} err="failed to get container status \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": rpc error: code = NotFound desc = could not find container \"a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64\": container with ID starting with a8f6a0e7dcf65ea34882b679ec34f34f96858264e42be16d351fbfcd863dcd64 not found: ID does not exist"
Sep 10 23:39:37.190521 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.190240    2575 scope.go:117] "RemoveContainer" containerID="fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"
Sep 10 23:39:37.190782 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.190550    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353"} err="failed to get container status \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": rpc error: code = NotFound desc = could not find container \"fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353\": container with ID starting with fc99fd33dfdd04bdce1c00d404807796f923de82482853b1b44e94ad4d23c353 not found: ID does not exist"
Sep 10 23:39:37.190782 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.190570    2575 scope.go:117] "RemoveContainer" containerID="d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"
Sep 10 23:39:37.191159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.190956    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499"} err="failed to get container status \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": rpc error: code = NotFound desc = could not find container \"d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499\": container with ID starting with d90abb1e185b70dd248346076a3576027e7eb8a49da98ddb91b886bfdff7c499 not found: ID does not exist"
Sep 10 23:39:37.191911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.191889    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 23:39:37.194058 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.194035    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 23:39:37.200060 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.200036    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:37.302998 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.302961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303098    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303164    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmr9p\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-kube-api-access-pmr9p\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303465    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303553 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.303817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.303610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404723 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404723 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404723 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404937 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404937 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404856    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404937 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.404937 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.404980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405064    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405122 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmr9p\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-kube-api-access-pmr9p\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405744 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405744 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405673    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.405744 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.405734    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.406257 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.406188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.407170 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.407144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408736 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408717    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408791 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.408885 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.408824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.409182 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.409125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.409387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.409331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.409731 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.409711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.410793 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.410774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.411028 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.410995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.411123 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.411052    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.411241 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.411224    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-config\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.411596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.411579    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.415858 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.415838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmr9p\" (UniqueName: \"kubernetes.io/projected/ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82-kube-api-access-pmr9p\") pod \"prometheus-k8s-0\" (UID: \"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.495487 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.495462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:39:37.781195 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:37.781116    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:39:37.784479 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:39:37.784457    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1f0dea_72aa_43a3_bbd7_08cb9ac2df82.slice/crio-feca24551503e5b1f284358ed2d9951009c2e1736e1326fa84ab4995d7222a90 WatchSource:0}: Error finding container feca24551503e5b1f284358ed2d9951009c2e1736e1326fa84ab4995d7222a90: Status 404 returned error can't find the container with id feca24551503e5b1f284358ed2d9951009c2e1736e1326fa84ab4995d7222a90
Sep 10 23:39:38.130688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:38.130607    2575 generic.go:356] "Generic (PLEG): container finished" podID="ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82" containerID="f9ab1b09395b080be2bcc51976c8879e774d9b7e5023a09b5574a7fbc8d137cb" exitCode=0
Sep 10 23:39:38.131124 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:38.130696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerDied","Data":"f9ab1b09395b080be2bcc51976c8879e774d9b7e5023a09b5574a7fbc8d137cb"}
Sep 10 23:39:38.131124 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:38.130788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"feca24551503e5b1f284358ed2d9951009c2e1736e1326fa84ab4995d7222a90"}
Sep 10 23:39:38.229608 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:38.228472    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c26bba8-37f8-4f01-971d-0b356b6fef53" path="/var/lib/kubelet/pods/2c26bba8-37f8-4f01-971d-0b356b6fef53/volumes"
Sep 10 23:39:39.139778 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139744    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"b543fa86dd020f4d49c8d7f09eb7763994c493610d3a07c05e1a5ee6b548b65a"}
Sep 10 23:39:39.140120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"175a8a76b935d177e979d7e71e5003db9cae8833ad61a30d7c3830c12ffc1115"}
Sep 10 23:39:39.140120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"5d11761aca4802f30be8d37be0701a51b9adee735a20e199f8556f5eb0f41366"}
Sep 10 23:39:39.140120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"2a9a137e2085d324be4bffe1048a679ed808cab564e66bb964bd2f8de562198b"}
Sep 10 23:39:39.140120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"0d3809226e98029ac33055b840b6df9e5cc4fd8b02dd86ccbcbf54208ea6e08e"}
Sep 10 23:39:39.140120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.139909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82","Type":"ContainerStarted","Data":"bb3ae78b61cd5f99b6043462ec8bf247b7f1a11859c71c0b802d7d9d65e1261a"}
Sep 10 23:39:39.166306 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:39.166249    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.147111901 podStartE2EDuration="2.166227611s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.11571ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:39.164421577 +0000 UTC m=+169.430418462" watchObservedRunningTime="2026-09-10 23:39:39.166227611 +0000 UTC m=+169.432224484"
Sep 10 23:39:42.495900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:39:42.495811    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:40:37.495612 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:40:37.495573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:40:37.512030 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:40:37.512004    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:40:38.418470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:40:38.418439    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:41:46.572913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.572876    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"]
Sep 10 23:41:46.575616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.575595    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.578125 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.578097    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 23:41:46.578125 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.578101    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 23:41:46.578307 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.578101    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x5ggk\""
Sep 10 23:41:46.578369 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.578312    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 23:41:46.578478 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.578457    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 23:41:46.595718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.595694    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"]
Sep 10 23:41:46.711076 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.711042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.711076 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.711081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xps28\" (UniqueName: \"kubernetes.io/projected/f0025c96-154c-491b-ac9d-e2bf68fca28d-kube-api-access-xps28\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.711286 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.711162    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.711286 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.711192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-manifests\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.711286 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.711209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-tmp\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812239 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812404 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-manifests\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812404 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-tmp\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812404 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812404 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xps28\" (UniqueName: \"kubernetes.io/projected/f0025c96-154c-491b-ac9d-e2bf68fca28d-kube-api-access-xps28\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.812744 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.812717    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-manifests\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.814688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.814657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0025c96-154c-491b-ac9d-e2bf68fca28d-tmp\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.814928 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.814907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.814969 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.814942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0025c96-154c-491b-ac9d-e2bf68fca28d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.821593 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.821570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xps28\" (UniqueName: \"kubernetes.io/projected/f0025c96-154c-491b-ac9d-e2bf68fca28d-kube-api-access-xps28\") pod \"opendatahub-operator-controller-manager-f676b6c5d-9sgns\" (UID: \"f0025c96-154c-491b-ac9d-e2bf68fca28d\") " pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:46.886151 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:46.886064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:47.171426 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:47.171348    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"]
Sep 10 23:41:47.175305 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:41:47.175278    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0025c96_154c_491b_ac9d_e2bf68fca28d.slice/crio-6a5690f2bc59215124636910b8638eddce134a73a56794a9e25a810c21ab1c93 WatchSource:0}: Error finding container 6a5690f2bc59215124636910b8638eddce134a73a56794a9e25a810c21ab1c93: Status 404 returned error can't find the container with id 6a5690f2bc59215124636910b8638eddce134a73a56794a9e25a810c21ab1c93
Sep 10 23:41:47.689163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:47.689117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns" event={"ID":"f0025c96-154c-491b-ac9d-e2bf68fca28d","Type":"ContainerStarted","Data":"6a5690f2bc59215124636910b8638eddce134a73a56794a9e25a810c21ab1c93"}
Sep 10 23:41:50.105113 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:50.105086    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 23:41:50.705772 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:50.705727    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns" event={"ID":"f0025c96-154c-491b-ac9d-e2bf68fca28d","Type":"ContainerStarted","Data":"4bcb71511401a012cc96f336eb764979123f535088dec8f03e33e85437132e5b"}
Sep 10 23:41:51.711880 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:51.711847    2575 generic.go:356] "Generic (PLEG): container finished" podID="f0025c96-154c-491b-ac9d-e2bf68fca28d" containerID="4bcb71511401a012cc96f336eb764979123f535088dec8f03e33e85437132e5b" exitCode=0
Sep 10 23:41:51.712271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:51.711917    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns" event={"ID":"f0025c96-154c-491b-ac9d-e2bf68fca28d","Type":"ContainerDied","Data":"4bcb71511401a012cc96f336eb764979123f535088dec8f03e33e85437132e5b"}
Sep 10 23:41:51.712411 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:51.712397    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:41:52.719357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:52.719315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns" event={"ID":"f0025c96-154c-491b-ac9d-e2bf68fca28d","Type":"ContainerStarted","Data":"7a0436c38d8f7619ab8bbf87f30705f45908abde5e5f9773e7ac3d66bd25546c"}
Sep 10 23:41:52.719752 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:52.719448    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:41:52.765204 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:41:52.765155    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns" podStartSLOduration=2.456506519 podStartE2EDuration="6.765140937s" totalImagesPullingTime="3.274077057s" totalInitContainerRuntime="1.034557361s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:41:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:41:52.755770134 +0000 UTC m=+303.021767016" watchObservedRunningTime="2026-09-10 23:41:52.765140937 +0000 UTC m=+303.031137807"
Sep 10 23:42:03.727125 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:03.727094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f676b6c5d-9sgns"
Sep 10 23:42:04.318324 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.318292    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"]
Sep 10 23:42:04.328851 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.328821    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.330522 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.330480    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"]
Sep 10 23:42:04.331527 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.331482    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:42:04.331763 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.331742    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 23:42:04.332340 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.332317    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 23:42:04.332544 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.332520    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 23:42:04.332657 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.332600    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 23:42:04.335632 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.335612    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bllhr\""
Sep 10 23:42:04.372361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.372336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.372470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.372366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7vgq\" (UniqueName: \"kubernetes.io/projected/010a610b-54c4-4b9d-b3e9-09b74c9de517-kube-api-access-g7vgq\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.372470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.372428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/010a610b-54c4-4b9d-b3e9-09b74c9de517-manager-config\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.372470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.372456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-metrics-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.473342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.473316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/010a610b-54c4-4b9d-b3e9-09b74c9de517-manager-config\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.473509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.473357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-metrics-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.473509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.473413    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.473611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.473541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7vgq\" (UniqueName: \"kubernetes.io/projected/010a610b-54c4-4b9d-b3e9-09b74c9de517-kube-api-access-g7vgq\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.474161 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.474087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/010a610b-54c4-4b9d-b3e9-09b74c9de517-manager-config\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.475998 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.475978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.476074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.476013    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/010a610b-54c4-4b9d-b3e9-09b74c9de517-metrics-cert\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.481905 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.481881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7vgq\" (UniqueName: \"kubernetes.io/projected/010a610b-54c4-4b9d-b3e9-09b74c9de517-kube-api-access-g7vgq\") pod \"lws-controller-manager-f59994447-7s9v8\" (UID: \"010a610b-54c4-4b9d-b3e9-09b74c9de517\") " pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.639681 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.639588    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:04.918836 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:04.918812    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"]
Sep 10 23:42:04.921470 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:42:04.921443    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010a610b_54c4_4b9d_b3e9_09b74c9de517.slice/crio-05a1bdf95c9e943ac905f411061c999d7097a0e404e55a2ec029bd18684b03c8 WatchSource:0}: Error finding container 05a1bdf95c9e943ac905f411061c999d7097a0e404e55a2ec029bd18684b03c8: Status 404 returned error can't find the container with id 05a1bdf95c9e943ac905f411061c999d7097a0e404e55a2ec029bd18684b03c8
Sep 10 23:42:05.782023 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:05.781979    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8" event={"ID":"010a610b-54c4-4b9d-b3e9-09b74c9de517","Type":"ContainerStarted","Data":"05a1bdf95c9e943ac905f411061c999d7097a0e404e55a2ec029bd18684b03c8"}
Sep 10 23:42:07.794050 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:07.794013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8" event={"ID":"010a610b-54c4-4b9d-b3e9-09b74c9de517","Type":"ContainerStarted","Data":"43836c04c992c902ec501f2e6dd7cbeee037d5e2229d83360710e37f882289a7"}
Sep 10 23:42:07.794470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:07.794144    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:42:07.810144 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:07.810095    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8" podStartSLOduration=1.385879648 podStartE2EDuration="3.810077428s" totalImagesPullingTime="2.42419778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:42:07.808401894 +0000 UTC m=+318.074398776" watchObservedRunningTime="2026-09-10 23:42:07.810077428 +0000 UTC m=+318.076074300"
Sep 10 23:42:08.600701 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.600668    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"]
Sep 10 23:42:08.604091 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.604072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.606605 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.606583    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 23:42:08.606960 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.606941    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 23:42:08.607427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.607410    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s5q4r\""
Sep 10 23:42:08.617569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.617545    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"]
Sep 10 23:42:08.715780 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.715750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/420ea471-84d8-42c0-9ba1-c60ad2564afb-tmp\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.715933 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.715784    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmnk\" (UniqueName: \"kubernetes.io/projected/420ea471-84d8-42c0-9ba1-c60ad2564afb-kube-api-access-zkmnk\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.715933 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.715841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420ea471-84d8-42c0-9ba1-c60ad2564afb-tls-certs\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.816811 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.816790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/420ea471-84d8-42c0-9ba1-c60ad2564afb-tmp\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.817265 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.816817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmnk\" (UniqueName: \"kubernetes.io/projected/420ea471-84d8-42c0-9ba1-c60ad2564afb-kube-api-access-zkmnk\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.817265 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.816839    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420ea471-84d8-42c0-9ba1-c60ad2564afb-tls-certs\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.819277 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.819245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/420ea471-84d8-42c0-9ba1-c60ad2564afb-tmp\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.819476 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.819460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420ea471-84d8-42c0-9ba1-c60ad2564afb-tls-certs\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.825160 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.825141    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmnk\" (UniqueName: \"kubernetes.io/projected/420ea471-84d8-42c0-9ba1-c60ad2564afb-kube-api-access-zkmnk\") pod \"kube-auth-proxy-7d44b74856-nxglv\" (UID: \"420ea471-84d8-42c0-9ba1-c60ad2564afb\") " pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:08.914842 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:08.914756    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"
Sep 10 23:42:09.186453 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:09.186379    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44b74856-nxglv"]
Sep 10 23:42:09.189742 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:42:09.189707    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420ea471_84d8_42c0_9ba1_c60ad2564afb.slice/crio-606256b89ff14ce451bafdd709605ac80d7a8a69e05a95eaef477306a24a00c8 WatchSource:0}: Error finding container 606256b89ff14ce451bafdd709605ac80d7a8a69e05a95eaef477306a24a00c8: Status 404 returned error can't find the container with id 606256b89ff14ce451bafdd709605ac80d7a8a69e05a95eaef477306a24a00c8
Sep 10 23:42:09.806982 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:09.806925    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv" event={"ID":"420ea471-84d8-42c0-9ba1-c60ad2564afb","Type":"ContainerStarted","Data":"606256b89ff14ce451bafdd709605ac80d7a8a69e05a95eaef477306a24a00c8"}
Sep 10 23:42:12.824565 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:12.824529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv" event={"ID":"420ea471-84d8-42c0-9ba1-c60ad2564afb","Type":"ContainerStarted","Data":"8c93ba8815650047c074cbfbe60f76d4b43bf5e63a99d3f1bc6873edba6e30ea"}
Sep 10 23:42:12.851347 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:12.851291    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d44b74856-nxglv" podStartSLOduration=2.2223708110000002 podStartE2EDuration="4.851277099s" totalImagesPullingTime="2.628906288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:42:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:42:12.849775695 +0000 UTC m=+323.115772587" watchObservedRunningTime="2026-09-10 23:42:12.851277099 +0000 UTC m=+323.117273970"
Sep 10 23:42:18.801916 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:42:18.801884    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f59994447-7s9v8"
Sep 10 23:43:46.118242 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.118204    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"]
Sep 10 23:43:46.126275 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.126249    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:46.128903 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.128877    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 23:43:46.129035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.128943    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 23:43:46.129377 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.129356    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 23:43:46.129777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.129760    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2z4sh\""
Sep 10 23:43:46.135760 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.135736    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"]
Sep 10 23:43:46.246041 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.246004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p86f\" (UniqueName: \"kubernetes.io/projected/dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1-kube-api-access-7p86f\") pod \"dns-operator-controller-manager-7b84cd7c9c-mxjkt\" (UID: \"dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:46.347035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.347006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p86f\" (UniqueName: \"kubernetes.io/projected/dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1-kube-api-access-7p86f\") pod \"dns-operator-controller-manager-7b84cd7c9c-mxjkt\" (UID: \"dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:46.357403 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.357380    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p86f\" (UniqueName: \"kubernetes.io/projected/dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1-kube-api-access-7p86f\") pod \"dns-operator-controller-manager-7b84cd7c9c-mxjkt\" (UID: \"dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:46.437203 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.437133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:46.719890 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:46.719865    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"]
Sep 10 23:43:46.721854 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:43:46.721828    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc32c78_6c4b_4e26_ad7a_7bcfe4bd6de1.slice/crio-429ce5e9f53c4ce7c5b3ff70cb3a8fe71ce285fb7cc752906677a0489018f82d WatchSource:0}: Error finding container 429ce5e9f53c4ce7c5b3ff70cb3a8fe71ce285fb7cc752906677a0489018f82d: Status 404 returned error can't find the container with id 429ce5e9f53c4ce7c5b3ff70cb3a8fe71ce285fb7cc752906677a0489018f82d
Sep 10 23:43:47.252435 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:47.252397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt" event={"ID":"dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1","Type":"ContainerStarted","Data":"429ce5e9f53c4ce7c5b3ff70cb3a8fe71ce285fb7cc752906677a0489018f82d"}
Sep 10 23:43:50.268311 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.268275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt" event={"ID":"dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1","Type":"ContainerStarted","Data":"baafb22f5319e0be81c9380f55bea715c5d36c4986a05a69452e3c6ddd16a6e9"}
Sep 10 23:43:50.268757 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.268397    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:43:50.288265 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.288212    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt" podStartSLOduration=1.789317762 podStartE2EDuration="4.288197931s" totalImagesPullingTime="2.498880169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:43:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:43:50.286559332 +0000 UTC m=+420.552556214" watchObservedRunningTime="2026-09-10 23:43:50.288197931 +0000 UTC m=+420.554194802"
Sep 10 23:43:50.809318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.809282    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"]
Sep 10 23:43:50.812985 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.812969    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.816041 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.816019    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 23:43:50.816189 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.816023    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 23:43:50.816189 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.816182    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mxc26\""
Sep 10 23:43:50.826154 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.826133    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"]
Sep 10 23:43:50.888469 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.888430    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgb7t\" (UniqueName: \"kubernetes.io/projected/c241d1ed-366e-41a7-ba86-57d34ce1847d-kube-api-access-zgb7t\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.888469 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.888470    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c241d1ed-366e-41a7-ba86-57d34ce1847d-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.888743 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.888698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.989538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.989480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.989538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.989540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgb7t\" (UniqueName: \"kubernetes.io/projected/c241d1ed-366e-41a7-ba86-57d34ce1847d-kube-api-access-zgb7t\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.989777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.989564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c241d1ed-366e-41a7-ba86-57d34ce1847d-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.989777 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:43:50.989638    2575 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 23:43:50.989777 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:43:50.989717    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert podName:c241d1ed-366e-41a7-ba86-57d34ce1847d nodeName:}" failed. No retries permitted until 2026-09-10 23:43:51.48969506 +0000 UTC m=+421.755691925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-kczh4" (UID: "c241d1ed-366e-41a7-ba86-57d34ce1847d") : secret "plugin-serving-cert" not found
Sep 10 23:43:50.990447 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.990428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c241d1ed-366e-41a7-ba86-57d34ce1847d-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:50.999173 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:50.999147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgb7t\" (UniqueName: \"kubernetes.io/projected/c241d1ed-366e-41a7-ba86-57d34ce1847d-kube-api-access-zgb7t\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:51.493453 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:51.493418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:51.495902 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:51.495880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c241d1ed-366e-41a7-ba86-57d34ce1847d-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kczh4\" (UID: \"c241d1ed-366e-41a7-ba86-57d34ce1847d\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:51.732837 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:51.732803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"
Sep 10 23:43:52.014813 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:52.014782    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4"]
Sep 10 23:43:52.017161 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:43:52.017121    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc241d1ed_366e_41a7_ba86_57d34ce1847d.slice/crio-f87347947b862b40c06c0d681bb1fb55e6b3abc8552b7d172b635917a6804077 WatchSource:0}: Error finding container f87347947b862b40c06c0d681bb1fb55e6b3abc8552b7d172b635917a6804077: Status 404 returned error can't find the container with id f87347947b862b40c06c0d681bb1fb55e6b3abc8552b7d172b635917a6804077
Sep 10 23:43:52.278903 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:52.278820    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4" event={"ID":"c241d1ed-366e-41a7-ba86-57d34ce1847d","Type":"ContainerStarted","Data":"f87347947b862b40c06c0d681bb1fb55e6b3abc8552b7d172b635917a6804077"}
Sep 10 23:43:57.318670 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:57.318635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4" event={"ID":"c241d1ed-366e-41a7-ba86-57d34ce1847d","Type":"ContainerStarted","Data":"465e6c00026f772cb5b9b4daba4ba60520ccb99d5a6ebea42fd542a3cd5ea85f"}
Sep 10 23:43:57.341370 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:43:57.341321    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kczh4" podStartSLOduration=2.9595363260000003 podStartE2EDuration="7.341305869s" totalImagesPullingTime="4.381769543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:43:57.339005324 +0000 UTC m=+427.605002193" watchObservedRunningTime="2026-09-10 23:43:57.341305869 +0000 UTC m=+427.607302738"
Sep 10 23:44:01.147161 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.147133    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"]
Sep 10 23:44:01.151254 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.151237    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.154341 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.154323    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-55njz\""
Sep 10 23:44:01.184303 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.184283    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"]
Sep 10 23:44:01.275306 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.275278    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mxjkt"
Sep 10 23:44:01.282731 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.282701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.282856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.282772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cr58\" (UniqueName: \"kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.384263 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.384229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cr58\" (UniqueName: \"kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.384422 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.384298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.384717 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.384700    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.394671 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.394645    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cr58\" (UniqueName: \"kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58\") pod \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.461385 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.461325    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:01.739736 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.739704    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"]
Sep 10 23:44:01.741339 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:44:01.741302    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c02260_fb4b_4349_821f_469dd577174f.slice/crio-7256be0d05f5fad67d0efcf5425d1773c172d4daf930bd092b92c00ca9f4c0fb WatchSource:0}: Error finding container 7256be0d05f5fad67d0efcf5425d1773c172d4daf930bd092b92c00ca9f4c0fb: Status 404 returned error can't find the container with id 7256be0d05f5fad67d0efcf5425d1773c172d4daf930bd092b92c00ca9f4c0fb
Sep 10 23:44:01.752074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.752050    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"]
Sep 10 23:44:01.761486 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.761467    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"]
Sep 10 23:44:01.800384 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.800365    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:01.805086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.805071    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:01.831331 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.831306    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:01.889365 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.889336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:01.889538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.889379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9gj\" (UniqueName: \"kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:01.990814 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.990735    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:01.990814 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.990783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9gj\" (UniqueName: \"kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:01.991136 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:01.991115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:02.007935 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:02.007909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9gj\" (UniqueName: \"kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj\") pod \"kuadrant-operator-controller-manager-5f685c496f-7ncf5\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:02.116304 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:02.116272    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:02.390648 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:02.390624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:02.392947 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:44:02.392923    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb44fc1c_2c06_483b_a4e1_ec3f7833518c.slice/crio-3cbd7c5beb579769010dbfb1391cda107a569947fb2552e087cf9844c5ca98de WatchSource:0}: Error finding container 3cbd7c5beb579769010dbfb1391cda107a569947fb2552e087cf9844c5ca98de: Status 404 returned error can't find the container with id 3cbd7c5beb579769010dbfb1391cda107a569947fb2552e087cf9844c5ca98de
Sep 10 23:44:03.351730 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:03.351688    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" event={"ID":"db44fc1c-2c06-483b-a4e1-ec3f7833518c","Type":"ContainerStarted","Data":"3cbd7c5beb579769010dbfb1391cda107a569947fb2552e087cf9844c5ca98de"}
Sep 10 23:44:06.371180 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.371140    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" event={"ID":"db44fc1c-2c06-483b-a4e1-ec3f7833518c","Type":"ContainerStarted","Data":"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"}
Sep 10 23:44:06.371599 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.371207    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:06.372942 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.372895    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g" podUID="83c02260-fb4b-4349-821f-469dd577174f" containerName="manager" containerID="cri-o://244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef" gracePeriod=2
Sep 10 23:44:06.397709 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.397667    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" podStartSLOduration=2.025865299 podStartE2EDuration="5.397650659s" totalImagesPullingTime="3.37178536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:44:06.396106711 +0000 UTC m=+436.662103591" watchObservedRunningTime="2026-09-10 23:44:06.397650659 +0000 UTC m=+436.663647529"
Sep 10 23:44:06.611270 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.611243    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:06.613354 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:44:06.613331    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="83c02260-fb4b-4349-821f-469dd577174f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:06.735737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.735641    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cr58\" (UniqueName: \"kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58\") pod \"83c02260-fb4b-4349-821f-469dd577174f\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") "
Sep 10 23:44:06.735896 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.735759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume\") pod \"83c02260-fb4b-4349-821f-469dd577174f\" (UID: \"83c02260-fb4b-4349-821f-469dd577174f\") "
Sep 10 23:44:06.736007 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.735979    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "83c02260-fb4b-4349-821f-469dd577174f" (UID: "83c02260-fb4b-4349-821f-469dd577174f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:44:06.737937 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.737918    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58" (OuterVolumeSpecName: "kube-api-access-7cr58") pod "83c02260-fb4b-4349-821f-469dd577174f" (UID: "83c02260-fb4b-4349-821f-469dd577174f"). InnerVolumeSpecName "kube-api-access-7cr58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:44:06.837314 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.837284    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cr58\" (UniqueName: \"kubernetes.io/projected/83c02260-fb4b-4349-821f-469dd577174f-kube-api-access-7cr58\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:44:06.837314 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:06.837310    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c02260-fb4b-4349-821f-469dd577174f-extensions-socket-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:44:07.379139 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:07.379104    2575 generic.go:356] "Generic (PLEG): container finished" podID="83c02260-fb4b-4349-821f-469dd577174f" containerID="244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef" exitCode=2
Sep 10 23:44:07.379622 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:07.379156    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:07.379622 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:07.379203    2575 scope.go:117] "RemoveContainer" containerID="244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef"
Sep 10 23:44:07.381191 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:44:07.381162    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="83c02260-fb4b-4349-821f-469dd577174f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:07.387990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:07.387971    2575 scope.go:117] "RemoveContainer" containerID="244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef"
Sep 10 23:44:07.388279 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:44:07.388259    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef\": container with ID starting with 244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef not found: ID does not exist" containerID="244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef"
Sep 10 23:44:07.388345 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:07.388291    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef"} err="failed to get container status \"244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef\": rpc error: code = NotFound desc = could not find container \"244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef\": container with ID starting with 244c29d4cd356565423a450384b85a9382149dcf0b9027537feaf440cbebe9ef not found: ID does not exist"
Sep 10 23:44:07.388879 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:44:07.388859    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-p5j2g\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="83c02260-fb4b-4349-821f-469dd577174f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-p5j2g"
Sep 10 23:44:08.227651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:08.227618    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c02260-fb4b-4349-821f-469dd577174f" path="/var/lib/kubelet/pods/83c02260-fb4b-4349-821f-469dd577174f/volumes"
Sep 10 23:44:17.382109 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:17.382022    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:29.782695 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:29.782662    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:29.783145 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:29.782928    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" podUID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" containerName="manager" containerID="cri-o://cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200" gracePeriod=10
Sep 10 23:44:30.031717 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.031695    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:30.144156 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.144079    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume\") pod \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") "
Sep 10 23:44:30.144156 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.144127    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9gj\" (UniqueName: \"kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj\") pod \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\" (UID: \"db44fc1c-2c06-483b-a4e1-ec3f7833518c\") "
Sep 10 23:44:30.144572 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.144549    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "db44fc1c-2c06-483b-a4e1-ec3f7833518c" (UID: "db44fc1c-2c06-483b-a4e1-ec3f7833518c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:44:30.146233 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.146205    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj" (OuterVolumeSpecName: "kube-api-access-gx9gj") pod "db44fc1c-2c06-483b-a4e1-ec3f7833518c" (UID: "db44fc1c-2c06-483b-a4e1-ec3f7833518c"). InnerVolumeSpecName "kube-api-access-gx9gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:44:30.245008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.244977    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db44fc1c-2c06-483b-a4e1-ec3f7833518c-extensions-socket-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:44:30.245008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.245009    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9gj\" (UniqueName: \"kubernetes.io/projected/db44fc1c-2c06-483b-a4e1-ec3f7833518c-kube-api-access-gx9gj\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:44:30.497374 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.497288    2575 generic.go:356] "Generic (PLEG): container finished" podID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" containerID="cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200" exitCode=0
Sep 10 23:44:30.497374 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.497330    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"
Sep 10 23:44:30.497617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.497368    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" event={"ID":"db44fc1c-2c06-483b-a4e1-ec3f7833518c","Type":"ContainerDied","Data":"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"}
Sep 10 23:44:30.497617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.497418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5" event={"ID":"db44fc1c-2c06-483b-a4e1-ec3f7833518c","Type":"ContainerDied","Data":"3cbd7c5beb579769010dbfb1391cda107a569947fb2552e087cf9844c5ca98de"}
Sep 10 23:44:30.497617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.497439    2575 scope.go:117] "RemoveContainer" containerID="cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"
Sep 10 23:44:30.505892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.505876    2575 scope.go:117] "RemoveContainer" containerID="cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"
Sep 10 23:44:30.506183 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:44:30.506154    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200\": container with ID starting with cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200 not found: ID does not exist" containerID="cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"
Sep 10 23:44:30.506273 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.506180    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200"} err="failed to get container status \"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200\": rpc error: code = NotFound desc = could not find container \"cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200\": container with ID starting with cb7591f9df32b6160a378fe7747216b163e083a158389bfb52fb44aa12a85200 not found: ID does not exist"
Sep 10 23:44:30.517538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.517518    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:30.522087 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:30.522068    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7ncf5"]
Sep 10 23:44:32.227820 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:32.227787    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" path="/var/lib/kubelet/pods/db44fc1c-2c06-483b-a4e1-ec3f7833518c/volumes"
Sep 10 23:44:47.816164 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816123    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816519    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" containerName="manager"
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816531    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" containerName="manager"
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816545    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c02260-fb4b-4349-821f-469dd577174f" containerName="manager"
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816550    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c02260-fb4b-4349-821f-469dd577174f" containerName="manager"
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816614    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c02260-fb4b-4349-821f-469dd577174f" containerName="manager"
Sep 10 23:44:47.816696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.816620    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="db44fc1c-2c06-483b-a4e1-ec3f7833518c" containerName="manager"
Sep 10 23:44:47.819844 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.819828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:47.823083 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.823062    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 23:44:47.835803 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.835783    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:44:47.894805 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.894770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:47.894986 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.894878    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kx6p\" (UniqueName: \"kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:47.910704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.910676    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:44:47.996304 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.996270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kx6p\" (UniqueName: \"kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:47.996482 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.996350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:47.996910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:47.996889    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:48.008861 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:48.008834    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kx6p\" (UniqueName: \"kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p\") pod \"limitador-limitador-8588749977-6bqtt\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") " pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:48.130167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:48.130104    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:48.405196 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:48.405147    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:44:48.408198 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:44:48.408169    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafc69e7_1eea_4d46_9641_61e9eae9a4a8.slice/crio-05ac6c0ab10c02edebecf5047e33a26fd750611373b94ab0a850da088e802f0b WatchSource:0}: Error finding container 05ac6c0ab10c02edebecf5047e33a26fd750611373b94ab0a850da088e802f0b: Status 404 returned error can't find the container with id 05ac6c0ab10c02edebecf5047e33a26fd750611373b94ab0a850da088e802f0b
Sep 10 23:44:48.590270 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:48.590236    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" event={"ID":"aafc69e7-1eea-4d46-9641-61e9eae9a4a8","Type":"ContainerStarted","Data":"05ac6c0ab10c02edebecf5047e33a26fd750611373b94ab0a850da088e802f0b"}
Sep 10 23:44:49.596772 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:49.596740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" event={"ID":"aafc69e7-1eea-4d46-9641-61e9eae9a4a8","Type":"ContainerStarted","Data":"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"}
Sep 10 23:44:49.597103 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:49.596918    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:44:49.618751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:44:49.618705    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" podStartSLOduration=1.556805329 podStartE2EDuration="2.618687551s" totalImagesPullingTime="1.061882222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:44:49.61831476 +0000 UTC m=+479.884311648" watchObservedRunningTime="2026-09-10 23:44:49.618687551 +0000 UTC m=+479.884684423"
Sep 10 23:45:00.602673 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:00.602644    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:45:01.925150 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:01.925116    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:45:01.925596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:01.925285    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" podUID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" containerName="limitador" containerID="cri-o://97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40" gracePeriod=30
Sep 10 23:45:02.471319 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.471294    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:45:02.524141 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.524066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file\") pod \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") "
Sep 10 23:45:02.524272 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.524181    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kx6p\" (UniqueName: \"kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p\") pod \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\" (UID: \"aafc69e7-1eea-4d46-9641-61e9eae9a4a8\") "
Sep 10 23:45:02.524450 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.524429    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file" (OuterVolumeSpecName: "config-file") pod "aafc69e7-1eea-4d46-9641-61e9eae9a4a8" (UID: "aafc69e7-1eea-4d46-9641-61e9eae9a4a8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:45:02.526164 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.526140    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p" (OuterVolumeSpecName: "kube-api-access-9kx6p") pod "aafc69e7-1eea-4d46-9641-61e9eae9a4a8" (UID: "aafc69e7-1eea-4d46-9641-61e9eae9a4a8"). InnerVolumeSpecName "kube-api-access-9kx6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:45:02.625066 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.625035    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kx6p\" (UniqueName: \"kubernetes.io/projected/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-kube-api-access-9kx6p\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:02.625066 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.625060    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aafc69e7-1eea-4d46-9641-61e9eae9a4a8-config-file\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:02.660354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.660318    2575 generic.go:356] "Generic (PLEG): container finished" podID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" containerID="97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40" exitCode=0
Sep 10 23:45:02.660524 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.660365    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt"
Sep 10 23:45:02.660524 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.660373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" event={"ID":"aafc69e7-1eea-4d46-9641-61e9eae9a4a8","Type":"ContainerDied","Data":"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"}
Sep 10 23:45:02.660524 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.660425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6bqtt" event={"ID":"aafc69e7-1eea-4d46-9641-61e9eae9a4a8","Type":"ContainerDied","Data":"05ac6c0ab10c02edebecf5047e33a26fd750611373b94ab0a850da088e802f0b"}
Sep 10 23:45:02.660524 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.660448    2575 scope.go:117] "RemoveContainer" containerID="97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"
Sep 10 23:45:02.675355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.675321    2575 scope.go:117] "RemoveContainer" containerID="97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"
Sep 10 23:45:02.675751 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:02.675729    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40\": container with ID starting with 97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40 not found: ID does not exist" containerID="97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"
Sep 10 23:45:02.675828 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.675760    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40"} err="failed to get container status \"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40\": rpc error: code = NotFound desc = could not find container \"97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40\": container with ID starting with 97126478bd06e579cac95f0a581cf9e5ecf95e0c57cadf335d18ddeb934cdb40 not found: ID does not exist"
Sep 10 23:45:02.685782 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.685762    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:45:02.688990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:02.688961    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6bqtt"]
Sep 10 23:45:04.228368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:04.228335    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" path="/var/lib/kubelet/pods/aafc69e7-1eea-4d46-9641-61e9eae9a4a8/volumes"
Sep 10 23:45:05.720531 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.720485    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"]
Sep 10 23:45:05.720910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.720897    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" containerName="limitador"
Sep 10 23:45:05.720952 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.720913    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" containerName="limitador"
Sep 10 23:45:05.720986 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.720978    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aafc69e7-1eea-4d46-9641-61e9eae9a4a8" containerName="limitador"
Sep 10 23:45:05.725569 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.725545    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.728153 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.728132    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:45:05.728987 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.728962    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 23:45:05.728987 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.728978    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5nwwt\""
Sep 10 23:45:05.729155 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.729113    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 23:45:05.734088 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.734067    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"]
Sep 10 23:45:05.753684 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.753660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/292d3884-0456-450e-b188-93fcc10f2be7-data\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.753818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.753731    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szmp6\" (UniqueName: \"kubernetes.io/projected/292d3884-0456-450e-b188-93fcc10f2be7-kube-api-access-szmp6\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.854694 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.854663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/292d3884-0456-450e-b188-93fcc10f2be7-data\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.854876 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.854760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szmp6\" (UniqueName: \"kubernetes.io/projected/292d3884-0456-450e-b188-93fcc10f2be7-kube-api-access-szmp6\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.855065 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.855047    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/292d3884-0456-450e-b188-93fcc10f2be7-data\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:05.863761 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:05.863728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szmp6\" (UniqueName: \"kubernetes.io/projected/292d3884-0456-450e-b188-93fcc10f2be7-kube-api-access-szmp6\") pod \"postgres-69fb4594cf-fmz25\" (UID: \"292d3884-0456-450e-b188-93fcc10f2be7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:06.036163 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:06.036137    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:06.309107 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:06.309082    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"]
Sep 10 23:45:06.310981 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:45:06.310954    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292d3884_0456_450e_b188_93fcc10f2be7.slice/crio-66ab7647b2aa24a88b73b92ba813fc65c0a2077463fabaefb843ea79ae4de548 WatchSource:0}: Error finding container 66ab7647b2aa24a88b73b92ba813fc65c0a2077463fabaefb843ea79ae4de548: Status 404 returned error can't find the container with id 66ab7647b2aa24a88b73b92ba813fc65c0a2077463fabaefb843ea79ae4de548
Sep 10 23:45:06.681997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:06.681919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25" event={"ID":"292d3884-0456-450e-b188-93fcc10f2be7","Type":"ContainerStarted","Data":"66ab7647b2aa24a88b73b92ba813fc65c0a2077463fabaefb843ea79ae4de548"}
Sep 10 23:45:08.694752 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:08.694719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25" event={"ID":"292d3884-0456-450e-b188-93fcc10f2be7","Type":"ContainerStarted","Data":"594c55a0c7f945b6069f7ab036f3b5064254c4173604344877afe8447167d3c8"}
Sep 10 23:45:08.695114 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:08.694829    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:08.709873 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:08.709827    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25" podStartSLOduration=1.921418388 podStartE2EDuration="3.709809135s" totalImagesPullingTime="1.788390747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:08.709432526 +0000 UTC m=+498.975429412" watchObservedRunningTime="2026-09-10 23:45:08.709809135 +0000 UTC m=+498.975806005"
Sep 10 23:45:14.727830 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:14.727800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmz25"
Sep 10 23:45:15.260066 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.260037    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-q8rsn"]
Sep 10 23:45:15.263602 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.263586    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.265808 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.265789    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9nnd4\""
Sep 10 23:45:15.271195 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.271172    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-q8rsn"]
Sep 10 23:45:15.312017 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.311992    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-q8rsn"]
Sep 10 23:45:15.312245 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:15.312228    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hw47l], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-q8rsn" podUID="56d8569a-8c3c-441a-b7d4-0698f68e72be"
Sep 10 23:45:15.334713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.334687    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-jf4ps"]
Sep 10 23:45:15.337616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.337597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw47l\" (UniqueName: \"kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l\") pod \"authorino-557f876f69-q8rsn\" (UID: \"56d8569a-8c3c-441a-b7d4-0698f68e72be\") " pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.338189 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.338175    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.340342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.340323    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 23:45:15.346836 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.346817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-jf4ps"]
Sep 10 23:45:15.438367 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.438334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw47l\" (UniqueName: \"kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l\") pod \"authorino-557f876f69-q8rsn\" (UID: \"56d8569a-8c3c-441a-b7d4-0698f68e72be\") " pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.438538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.438398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.438538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.438459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97npr\" (UniqueName: \"kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.446123 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.446090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw47l\" (UniqueName: \"kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l\") pod \"authorino-557f876f69-q8rsn\" (UID: \"56d8569a-8c3c-441a-b7d4-0698f68e72be\") " pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.513791 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.513736    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-jf4ps"]
Sep 10 23:45:15.514006 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:15.513987    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-97npr tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-jf4ps" podUID="432c5f68-e88d-4158-a3e5-65481ddb9bff"
Sep 10 23:45:15.539276 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.539247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97npr\" (UniqueName: \"kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.539392 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.539349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.541753 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.541733    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.549934 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.549911    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97npr\" (UniqueName: \"kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr\") pod \"authorino-6d4dfc649f-jf4ps\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") " pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.729515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.729462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.729933 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.729462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.736999 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.736980    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:15.739738 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.739722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:15.842025 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.842001    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97npr\" (UniqueName: \"kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr\") pod \"432c5f68-e88d-4158-a3e5-65481ddb9bff\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") "
Sep 10 23:45:15.842137 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.842056    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert\") pod \"432c5f68-e88d-4158-a3e5-65481ddb9bff\" (UID: \"432c5f68-e88d-4158-a3e5-65481ddb9bff\") "
Sep 10 23:45:15.842137 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.842086    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw47l\" (UniqueName: \"kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l\") pod \"56d8569a-8c3c-441a-b7d4-0698f68e72be\" (UID: \"56d8569a-8c3c-441a-b7d4-0698f68e72be\") "
Sep 10 23:45:15.844253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.844226    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l" (OuterVolumeSpecName: "kube-api-access-hw47l") pod "56d8569a-8c3c-441a-b7d4-0698f68e72be" (UID: "56d8569a-8c3c-441a-b7d4-0698f68e72be"). InnerVolumeSpecName "kube-api-access-hw47l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:45:15.844346 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.844256    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr" (OuterVolumeSpecName: "kube-api-access-97npr") pod "432c5f68-e88d-4158-a3e5-65481ddb9bff" (UID: "432c5f68-e88d-4158-a3e5-65481ddb9bff"). InnerVolumeSpecName "kube-api-access-97npr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:45:15.844346 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.844323    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "432c5f68-e88d-4158-a3e5-65481ddb9bff" (UID: "432c5f68-e88d-4158-a3e5-65481ddb9bff"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:45:15.943483 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.943439    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97npr\" (UniqueName: \"kubernetes.io/projected/432c5f68-e88d-4158-a3e5-65481ddb9bff-kube-api-access-97npr\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:15.943483 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.943478    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/432c5f68-e88d-4158-a3e5-65481ddb9bff-tls-cert\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:15.943483 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:15.943514    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw47l\" (UniqueName: \"kubernetes.io/projected/56d8569a-8c3c-441a-b7d4-0698f68e72be-kube-api-access-hw47l\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:16.734256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.734221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q8rsn"
Sep 10 23:45:16.734715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.734221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-jf4ps"
Sep 10 23:45:16.763094 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.763070    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-q8rsn"]
Sep 10 23:45:16.769720 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.769693    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-q8rsn"]
Sep 10 23:45:16.788777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.788752    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-jf4ps"]
Sep 10 23:45:16.790700 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:16.790681    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-jf4ps"]
Sep 10 23:45:18.227779 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:18.227742    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432c5f68-e88d-4158-a3e5-65481ddb9bff" path="/var/lib/kubelet/pods/432c5f68-e88d-4158-a3e5-65481ddb9bff/volumes"
Sep 10 23:45:18.228133 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:18.227972    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56d8569a-8c3c-441a-b7d4-0698f68e72be" path="/var/lib/kubelet/pods/56d8569a-8c3c-441a-b7d4-0698f68e72be/volumes"
Sep 10 23:45:22.170559 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.170527    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:22.174344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.174328    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.176508 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.176473    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 23:45:22.176629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.176599    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 23:45:22.176740 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.176721    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8t9fk\""
Sep 10 23:45:22.176795 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.176758    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 23:45:22.183648 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.183625    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:22.297538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.297510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.297871 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.297854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.297914 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.297894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2qmt\" (UniqueName: \"kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.342067 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.342040    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 10 23:45:22.345391 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.345373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.354026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.354006    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 10 23:45:22.398824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.398973 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.398973 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.398973 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb4z9\" (UniqueName: \"kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.398973 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.399174 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.398995    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2qmt\" (UniqueName: \"kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.399174 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:22.399100    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 23:45:22.399174 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:22.399173    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs podName:42f44f00-1c62-43f2-8443-b4a1caff5c39 nodeName:}" failed. No retries permitted until 2026-09-10 23:45:22.8991534 +0000 UTC m=+513.165150249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs") pod "maas-controller-775bfcf6c9-trvzs" (UID: "42f44f00-1c62-43f2-8443-b4a1caff5c39") : secret "maas-controller-metrics-tls" not found
Sep 10 23:45:22.401332 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.401311    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.406934 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.406917    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2qmt\" (UniqueName: \"kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.499715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.499646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.499715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.499680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.499856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.499715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb4z9\" (UniqueName: \"kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.499856 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:22.499788    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 23:45:22.499856 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:22.499853    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs podName:5832202a-b15c-4121-be80-6044399ed222 nodeName:}" failed. No retries permitted until 2026-09-10 23:45:22.999836032 +0000 UTC m=+513.265832884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs") pod "maas-controller-6cf5467c5b-5g2q9" (UID: "5832202a-b15c-4121-be80-6044399ed222") : secret "maas-controller-metrics-tls" not found
Sep 10 23:45:22.502021 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.502002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.507233 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.507211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb4z9\" (UniqueName: \"kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:22.902997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.902963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:22.905424 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:22.905393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") pod \"maas-controller-775bfcf6c9-trvzs\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") " pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:23.003660 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.003630    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:23.005934 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.005917    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") pod \"maas-controller-6cf5467c5b-5g2q9\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") " pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:23.085286 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.085250    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:23.256356 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.256287    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:23.358038 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.358012    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:23.361003 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:45:23.360978    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f44f00_1c62_43f2_8443_b4a1caff5c39.slice/crio-c24bd25d4eb3e3e39cf4fc69026d9f2dcc5deab5448a3fb2ae15873eba319deb WatchSource:0}: Error finding container c24bd25d4eb3e3e39cf4fc69026d9f2dcc5deab5448a3fb2ae15873eba319deb: Status 404 returned error can't find the container with id c24bd25d4eb3e3e39cf4fc69026d9f2dcc5deab5448a3fb2ae15873eba319deb
Sep 10 23:45:23.530653 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:45:23.530626    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5832202a_b15c_4121_be80_6044399ed222.slice/crio-054ea17a1b02f9ff9865265bc84acddf8b0fd8b789d1a14cfc044e9f355cd47d WatchSource:0}: Error finding container 054ea17a1b02f9ff9865265bc84acddf8b0fd8b789d1a14cfc044e9f355cd47d: Status 404 returned error can't find the container with id 054ea17a1b02f9ff9865265bc84acddf8b0fd8b789d1a14cfc044e9f355cd47d
Sep 10 23:45:23.534815 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.534795    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 10 23:45:23.772864 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.772826    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" event={"ID":"42f44f00-1c62-43f2-8443-b4a1caff5c39","Type":"ContainerStarted","Data":"c24bd25d4eb3e3e39cf4fc69026d9f2dcc5deab5448a3fb2ae15873eba319deb"}
Sep 10 23:45:23.773924 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:23.773893    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" event={"ID":"5832202a-b15c-4121-be80-6044399ed222","Type":"ContainerStarted","Data":"054ea17a1b02f9ff9865265bc84acddf8b0fd8b789d1a14cfc044e9f355cd47d"}
Sep 10 23:45:26.792666 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.792630    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" event={"ID":"42f44f00-1c62-43f2-8443-b4a1caff5c39","Type":"ContainerStarted","Data":"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"}
Sep 10 23:45:26.793114 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.792695    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:26.794172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.794150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" event={"ID":"5832202a-b15c-4121-be80-6044399ed222","Type":"ContainerStarted","Data":"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"}
Sep 10 23:45:26.794285 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.794241    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:26.811378 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.811336    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" podStartSLOduration=2.142885102 podStartE2EDuration="4.811321063s" totalImagesPullingTime="2.668435961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:26.809375355 +0000 UTC m=+517.075372246" watchObservedRunningTime="2026-09-10 23:45:26.811321063 +0000 UTC m=+517.077317935"
Sep 10 23:45:26.826879 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:26.826839    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" podStartSLOduration=2.321123078 podStartE2EDuration="4.826830607s" totalImagesPullingTime="2.505707529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:26.826734034 +0000 UTC m=+517.092730929" watchObservedRunningTime="2026-09-10 23:45:26.826830607 +0000 UTC m=+517.092827475"
Sep 10 23:45:37.801114 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:37.801082    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 10 23:45:37.801552 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:37.801160    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:37.861631 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:37.861599    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:37.861808 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:37.861780    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" podUID="42f44f00-1c62-43f2-8443-b4a1caff5c39" containerName="manager" containerID="cri-o://e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d" gracePeriod=30
Sep 10 23:45:38.096761 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.096735    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:38.243813 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.243785    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2qmt\" (UniqueName: \"kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt\") pod \"42f44f00-1c62-43f2-8443-b4a1caff5c39\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") "
Sep 10 23:45:38.243976 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.243869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert\") pod \"42f44f00-1c62-43f2-8443-b4a1caff5c39\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") "
Sep 10 23:45:38.243976 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.243899    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") pod \"42f44f00-1c62-43f2-8443-b4a1caff5c39\" (UID: \"42f44f00-1c62-43f2-8443-b4a1caff5c39\") "
Sep 10 23:45:38.246046 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.246012    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "42f44f00-1c62-43f2-8443-b4a1caff5c39" (UID: "42f44f00-1c62-43f2-8443-b4a1caff5c39"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:45:38.246147 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.246079    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt" (OuterVolumeSpecName: "kube-api-access-h2qmt") pod "42f44f00-1c62-43f2-8443-b4a1caff5c39" (UID: "42f44f00-1c62-43f2-8443-b4a1caff5c39"). InnerVolumeSpecName "kube-api-access-h2qmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:45:38.246195 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.246147    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "42f44f00-1c62-43f2-8443-b4a1caff5c39" (UID: "42f44f00-1c62-43f2-8443-b4a1caff5c39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:45:38.344734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.344674    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2qmt\" (UniqueName: \"kubernetes.io/projected/42f44f00-1c62-43f2-8443-b4a1caff5c39-kube-api-access-h2qmt\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:38.344734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.344696    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-webhook-cert\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:38.344734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.344705    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f44f00-1c62-43f2-8443-b4a1caff5c39-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:45:38.855836 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.855798    2575 generic.go:356] "Generic (PLEG): container finished" podID="42f44f00-1c62-43f2-8443-b4a1caff5c39" containerID="e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d" exitCode=0
Sep 10 23:45:38.856232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.855849    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775bfcf6c9-trvzs"
Sep 10 23:45:38.856232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.855882    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" event={"ID":"42f44f00-1c62-43f2-8443-b4a1caff5c39","Type":"ContainerDied","Data":"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"}
Sep 10 23:45:38.856232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.855933    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775bfcf6c9-trvzs" event={"ID":"42f44f00-1c62-43f2-8443-b4a1caff5c39","Type":"ContainerDied","Data":"c24bd25d4eb3e3e39cf4fc69026d9f2dcc5deab5448a3fb2ae15873eba319deb"}
Sep 10 23:45:38.856232 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.855956    2575 scope.go:117] "RemoveContainer" containerID="e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"
Sep 10 23:45:38.865694 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.865448    2575 scope.go:117] "RemoveContainer" containerID="e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"
Sep 10 23:45:38.865934 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:45:38.865710    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d\": container with ID starting with e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d not found: ID does not exist" containerID="e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"
Sep 10 23:45:38.865934 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.865734    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d"} err="failed to get container status \"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d\": rpc error: code = NotFound desc = could not find container \"e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d\": container with ID starting with e4ccfa8f7d24dc2a9c4c042bcdd9fa24152a99bac58fc98a68976ebaad03466d not found: ID does not exist"
Sep 10 23:45:38.876449 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.876426    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:38.879752 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:38.879721    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-775bfcf6c9-trvzs"]
Sep 10 23:45:40.233472 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:40.233434    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f44f00-1c62-43f2-8443-b4a1caff5c39" path="/var/lib/kubelet/pods/42f44f00-1c62-43f2-8443-b4a1caff5c39/volumes"
Sep 10 23:45:59.898029 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.898000    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 10 23:45:59.898444 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.898427    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f44f00-1c62-43f2-8443-b4a1caff5c39" containerName="manager"
Sep 10 23:45:59.898522 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.898447    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f44f00-1c62-43f2-8443-b4a1caff5c39" containerName="manager"
Sep 10 23:45:59.898588 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.898556    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f44f00-1c62-43f2-8443-b4a1caff5c39" containerName="manager"
Sep 10 23:45:59.904328 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.904306    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:45:59.906544 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.906524    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 23:45:59.906669 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.906532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 23:45:59.913633 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.913611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 10 23:45:59.931693 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.931660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgp6c\" (UniqueName: \"kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:45:59.931838 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.931753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:45:59.931838 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.931775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:45:59.931953 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:45:59.931882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.032870 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.032835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.033045 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.032892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.033045 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.032937    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.033045 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.032974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgp6c\" (UniqueName: \"kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.033045 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:46:00.032988    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 23:46:00.033293 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:46:00.033055    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 23:46:00.033293 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:46:00.033066    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs podName:37282bc7-3e5b-4cd0-b358-b3970f7a84e2 nodeName:}" failed. No retries permitted until 2026-09-10 23:46:00.533048202 +0000 UTC m=+550.799045058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs") pod "maas-api-78cdb675f8-44ktm" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2") : secret "maas-api-metrics-tls" not found
Sep 10 23:46:00.033293 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:46:00.033131    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls podName:37282bc7-3e5b-4cd0-b358-b3970f7a84e2 nodeName:}" failed. No retries permitted until 2026-09-10 23:46:00.533113678 +0000 UTC m=+550.799110526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls") pod "maas-api-78cdb675f8-44ktm" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2") : secret "maas-api-serving-cert" not found
Sep 10 23:46:00.033700 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.033679    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.043670 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.043642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgp6c\" (UniqueName: \"kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.536988 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.536950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.536988 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.536991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.539483 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.539456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.539623 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.539522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") pod \"maas-api-78cdb675f8-44ktm\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") " pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.574920 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.574887    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"]
Sep 10 23:46:00.578346 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.578329    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.580684 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.580663    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qq8gw\""
Sep 10 23:46:00.580808 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.580666    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 23:46:00.589825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.589803    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"]
Sep 10 23:46:00.637732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.637694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81f5fb-b916-4501-ab64-28f1aa60dc30-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.637902 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.637772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brphz\" (UniqueName: \"kubernetes.io/projected/cc81f5fb-b916-4501-ab64-28f1aa60dc30-kube-api-access-brphz\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.738868 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.738837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81f5fb-b916-4501-ab64-28f1aa60dc30-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.739043 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.738914    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brphz\" (UniqueName: \"kubernetes.io/projected/cc81f5fb-b916-4501-ab64-28f1aa60dc30-kube-api-access-brphz\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.739537 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.739518    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81f5fb-b916-4501-ab64-28f1aa60dc30-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.746646 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.746626    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brphz\" (UniqueName: \"kubernetes.io/projected/cc81f5fb-b916-4501-ab64-28f1aa60dc30-kube-api-access-brphz\") pod \"payload-processing-5b54b7fbb7-qv9kd\" (UID: \"cc81f5fb-b916-4501-ab64-28f1aa60dc30\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:00.815568 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.815457    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:00.889824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:00.889797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:01.088537 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:01.088511    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 10 23:46:01.090554 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:46:01.090528    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37282bc7_3e5b_4cd0_b358_b3970f7a84e2.slice/crio-dc25a959dce462247db7db1bb73ae2b98a50ce18f6fd47c45f694934aab11ac7 WatchSource:0}: Error finding container dc25a959dce462247db7db1bb73ae2b98a50ce18f6fd47c45f694934aab11ac7: Status 404 returned error can't find the container with id dc25a959dce462247db7db1bb73ae2b98a50ce18f6fd47c45f694934aab11ac7
Sep 10 23:46:01.164300 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:01.164280    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"]
Sep 10 23:46:01.165924 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:46:01.165899    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc81f5fb_b916_4501_ab64_28f1aa60dc30.slice/crio-0fc9e7d8db949d943b96369de05952148f9b3ab71fa94a53763f17de497d64ad WatchSource:0}: Error finding container 0fc9e7d8db949d943b96369de05952148f9b3ab71fa94a53763f17de497d64ad: Status 404 returned error can't find the container with id 0fc9e7d8db949d943b96369de05952148f9b3ab71fa94a53763f17de497d64ad
Sep 10 23:46:01.972102 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:01.972063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd" event={"ID":"cc81f5fb-b916-4501-ab64-28f1aa60dc30","Type":"ContainerStarted","Data":"0fc9e7d8db949d943b96369de05952148f9b3ab71fa94a53763f17de497d64ad"}
Sep 10 23:46:01.974233 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:01.974148    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" event={"ID":"37282bc7-3e5b-4cd0-b358-b3970f7a84e2","Type":"ContainerStarted","Data":"dc25a959dce462247db7db1bb73ae2b98a50ce18f6fd47c45f694934aab11ac7"}
Sep 10 23:46:03.987477 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:03.987386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" event={"ID":"37282bc7-3e5b-4cd0-b358-b3970f7a84e2","Type":"ContainerStarted","Data":"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"}
Sep 10 23:46:03.987477 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:03.987473    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:03.989070 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:03.989046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd" event={"ID":"cc81f5fb-b916-4501-ab64-28f1aa60dc30","Type":"ContainerStarted","Data":"973c68f7956e8b98fce6e5e31dd8ac7c7bea66c8bded103ec1e82dfade43b29b"}
Sep 10 23:46:03.989195 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:03.989164    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:04.006721 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:04.006667    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" podStartSLOduration=2.520520764 podStartE2EDuration="5.00665004s" totalImagesPullingTime="2.486129276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:04.003832484 +0000 UTC m=+554.269829353" watchObservedRunningTime="2026-09-10 23:46:04.00665004 +0000 UTC m=+554.272646912"
Sep 10 23:46:04.021073 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:04.021028    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd" podStartSLOduration=1.5649911250000001 podStartE2EDuration="4.021013454s" totalImagesPullingTime="2.456022329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:04.019598963 +0000 UTC m=+554.285595852" watchObservedRunningTime="2026-09-10 23:46:04.021013454 +0000 UTC m=+554.287010324"
Sep 10 23:46:09.999055 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:09.999028    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 10 23:46:14.996409 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:14.996379    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qv9kd"
Sep 10 23:46:29.913008 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.912971    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"]
Sep 10 23:46:29.917342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.917320    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:29.919446 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.919420    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 23:46:29.920520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.920358    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rg6w4\""
Sep 10 23:46:29.920520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.920393    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 23:46:29.920520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.920393    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 23:46:29.927256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:29.927236    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"]
Sep 10 23:46:30.094748 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094717    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.094923 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.094923 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.094923 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x754t\" (UniqueName: \"kubernetes.io/projected/53558d1b-f3e7-4cb5-b9f0-62182e454e60-kube-api-access-x754t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.094923 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.095083 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.094947    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196371 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196371 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x754t\" (UniqueName: \"kubernetes.io/projected/53558d1b-f3e7-4cb5-b9f0-62182e454e60-kube-api-access-x754t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196371 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196336    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196371 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196413    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196869 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196766    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196869 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.196869 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.196854    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.199461 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.199435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53558d1b-f3e7-4cb5-b9f0-62182e454e60-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.199832 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.199813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53558d1b-f3e7-4cb5-b9f0-62182e454e60-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.205236 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.205213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x754t\" (UniqueName: \"kubernetes.io/projected/53558d1b-f3e7-4cb5-b9f0-62182e454e60-kube-api-access-x754t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-h6bgb\" (UID: \"53558d1b-f3e7-4cb5-b9f0-62182e454e60\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.231112 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.231091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:30.501413 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:30.501382    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"]
Sep 10 23:46:30.503696 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:46:30.503673    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53558d1b_f3e7_4cb5_b9f0_62182e454e60.slice/crio-49b7b92faa99e228bbef1189f5bb4ec80f80ca3e0db7ee3efd8a1bb5fefcbb79 WatchSource:0}: Error finding container 49b7b92faa99e228bbef1189f5bb4ec80f80ca3e0db7ee3efd8a1bb5fefcbb79: Status 404 returned error can't find the container with id 49b7b92faa99e228bbef1189f5bb4ec80f80ca3e0db7ee3efd8a1bb5fefcbb79
Sep 10 23:46:31.120277 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:31.120244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb" event={"ID":"53558d1b-f3e7-4cb5-b9f0-62182e454e60","Type":"ContainerStarted","Data":"49b7b92faa99e228bbef1189f5bb4ec80f80ca3e0db7ee3efd8a1bb5fefcbb79"}
Sep 10 23:46:32.126884 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:32.126809    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb" event={"ID":"53558d1b-f3e7-4cb5-b9f0-62182e454e60","Type":"ContainerStarted","Data":"fc851fc66ca3b20e9a4714e8fd03031db41fe370e31b8629752eadf433fa4a07"}
Sep 10 23:46:32.144485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:32.144440    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb" podStartSLOduration=1.821169239 podStartE2EDuration="3.144425503s" totalImagesPullingTime="1.323256264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:32.142450661 +0000 UTC m=+582.408447530" watchObservedRunningTime="2026-09-10 23:46:32.144425503 +0000 UTC m=+582.410422369"
Sep 10 23:46:40.231469 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:40.231443    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:40.231933 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:40.231487    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:40.244022 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:40.244001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:46:41.182711 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:46:41.182685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-h6bgb"
Sep 10 23:47:43.928267 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:43.928235    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"]
Sep 10 23:47:43.931756 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:43.931736    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:43.933802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:43.933784    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 23:47:43.940120 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:43.940099    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"]
Sep 10 23:47:44.042414 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.042574 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c758df08-adc3-4380-abcc-f9e5cdb64997-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.042574 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.042574 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.042574 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brgcn\" (UniqueName: \"kubernetes.io/projected/c758df08-adc3-4380-abcc-f9e5cdb64997-kube-api-access-brgcn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.042718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.042682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143280 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143245    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c758df08-adc3-4380-abcc-f9e5cdb64997-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143384    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143666 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brgcn\" (UniqueName: \"kubernetes.io/projected/c758df08-adc3-4380-abcc-f9e5cdb64997-kube-api-access-brgcn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143780 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143844 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.143844 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.143829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.145627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.145609    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c758df08-adc3-4380-abcc-f9e5cdb64997-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.145984 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.145966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c758df08-adc3-4380-abcc-f9e5cdb64997-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.151362 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.151334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brgcn\" (UniqueName: \"kubernetes.io/projected/c758df08-adc3-4380-abcc-f9e5cdb64997-kube-api-access-brgcn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6\" (UID: \"c758df08-adc3-4380-abcc-f9e5cdb64997\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.241404 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.241332    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:44.511533 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.511483    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"]
Sep 10 23:47:44.513782 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:47:44.513754    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc758df08_adc3_4380_abcc_f9e5cdb64997.slice/crio-8c1aaf0ebe453b840c50135e393d9163738a414e44d9daa627c7feee20d3c2bb WatchSource:0}: Error finding container 8c1aaf0ebe453b840c50135e393d9163738a414e44d9daa627c7feee20d3c2bb: Status 404 returned error can't find the container with id 8c1aaf0ebe453b840c50135e393d9163738a414e44d9daa627c7feee20d3c2bb
Sep 10 23:47:44.515063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:44.515046    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:47:45.486236 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:45.486203    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6" event={"ID":"c758df08-adc3-4380-abcc-f9e5cdb64997","Type":"ContainerStarted","Data":"45f74b1b1c8284a840271aca05c7b102527631291a85fbcce9034dea2429ac4e"}
Sep 10 23:47:45.486236 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:45.486242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6" event={"ID":"c758df08-adc3-4380-abcc-f9e5cdb64997","Type":"ContainerStarted","Data":"8c1aaf0ebe453b840c50135e393d9163738a414e44d9daa627c7feee20d3c2bb"}
Sep 10 23:47:45.503202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:45.503158    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6" podStartSLOduration=2.313816961 podStartE2EDuration="2.503142617s" totalImagesPullingTime="189.325656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:47:45.5016189 +0000 UTC m=+655.767615770" watchObservedRunningTime="2026-09-10 23:47:45.503142617 +0000 UTC m=+655.769139487"
Sep 10 23:47:54.242043 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:54.242013    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:54.242433 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:54.242056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:54.254096 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:54.254075    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:47:54.544632 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:47:54.544604    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6"
Sep 10 23:49:50.420724 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.420692    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7850-4bafd0"]
Sep 10 23:49:50.424249 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.424232    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:50.426726 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.426706    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k9g2h\""
Sep 10 23:49:50.431798 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.431778    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7850-4bafd0"]
Sep 10 23:49:50.492917 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.492891    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4dc\" (UniqueName: \"kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc\") pod \"test-curl-7850-4bafd0\" (UID: \"c1f7f048-5029-4763-8fbb-b33258181953\") " pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:50.593620 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.593586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4dc\" (UniqueName: \"kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc\") pod \"test-curl-7850-4bafd0\" (UID: \"c1f7f048-5029-4763-8fbb-b33258181953\") " pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:50.602391 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.602369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4dc\" (UniqueName: \"kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc\") pod \"test-curl-7850-4bafd0\" (UID: \"c1f7f048-5029-4763-8fbb-b33258181953\") " pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:50.734849 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:50.734778    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:51.003797 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:51.003775    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7850-4bafd0"]
Sep 10 23:49:51.005871 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:49:51.005842    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1f7f048_5029_4763_8fbb_b33258181953.slice/crio-683e70a9c68cb57e4bf589138beb40d6c35dd314a19f3b42b734a056614ce61f WatchSource:0}: Error finding container 683e70a9c68cb57e4bf589138beb40d6c35dd314a19f3b42b734a056614ce61f: Status 404 returned error can't find the container with id 683e70a9c68cb57e4bf589138beb40d6c35dd314a19f3b42b734a056614ce61f
Sep 10 23:49:51.120334 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:51.120304    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7850-4bafd0" event={"ID":"c1f7f048-5029-4763-8fbb-b33258181953","Type":"ContainerStarted","Data":"683e70a9c68cb57e4bf589138beb40d6c35dd314a19f3b42b734a056614ce61f"}
Sep 10 23:49:53.640515 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.640468    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:49:53.652396 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.652374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.654967 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.654937    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ce1dd5-serving-cert\""
Sep 10 23:49:53.655797 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.655772    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h7jzs\""
Sep 10 23:49:53.658661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.658641    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:49:53.727188 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.727156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.727357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.727196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.727357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.727254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hq2\" (UniqueName: \"kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.727357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.727305    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.828520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.828479    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.828631 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.828553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.828631 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.828601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hq2\" (UniqueName: \"kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.828746 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.828644    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.829237 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.829215    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.830895 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.830876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.831000 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.830927    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.840162 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.840143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hq2\" (UniqueName: \"kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2\") pod \"maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:53.965462 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:53.965375    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:54.137465 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:54.137431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7850-4bafd0" event={"ID":"c1f7f048-5029-4763-8fbb-b33258181953","Type":"ContainerStarted","Data":"f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319"}
Sep 10 23:49:54.260610 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:54.260511    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7850-4bafd0" podStartSLOduration=1.5256784209999998 podStartE2EDuration="4.260477064s" totalImagesPullingTime="2.734798643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:49:54.166984104 +0000 UTC m=+784.432980973" watchObservedRunningTime="2026-09-10 23:49:54.260477064 +0000 UTC m=+784.526473934"
Sep 10 23:49:54.262920 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:54.262899    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:49:54.264784 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:49:54.264760    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb99fb18_3bc6_40d1_b4ab_99c3f58f3ef3.slice/crio-a001d6a15666cfea9d2a19c85745b9fe39a55c8b69b85dcd4759fe2b87841330 WatchSource:0}: Error finding container a001d6a15666cfea9d2a19c85745b9fe39a55c8b69b85dcd4759fe2b87841330: Status 404 returned error can't find the container with id a001d6a15666cfea9d2a19c85745b9fe39a55c8b69b85dcd4759fe2b87841330
Sep 10 23:49:54.899832 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:54.899805    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7850-4bafd0"]
Sep 10 23:49:54.903300 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:54.903245    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7850-4bafd0"]
Sep 10 23:49:55.144645 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:55.144603    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" event={"ID":"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3","Type":"ContainerStarted","Data":"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"}
Sep 10 23:49:55.144800 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:55.144704    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" event={"ID":"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3","Type":"ContainerStarted","Data":"a001d6a15666cfea9d2a19c85745b9fe39a55c8b69b85dcd4759fe2b87841330"}
Sep 10 23:49:55.144859 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:49:55.144823    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7850-4bafd0" container="test-curl-7850-4bafd0"
Sep 10 23:49:55.166674 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:55.166570    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" podStartSLOduration=1.710326819 podStartE2EDuration="2.166555508s" totalImagesPullingTime="456.228689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:49:55.165839519 +0000 UTC m=+785.431836390" watchObservedRunningTime="2026-09-10 23:49:55.166555508 +0000 UTC m=+785.432552378"
Sep 10 23:49:56.149807 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:56.149749    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7850-4bafd0" podUID="c1f7f048-5029-4763-8fbb-b33258181953" containerName="test-curl-7850-4bafd0" containerID="cri-o://f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319" gracePeriod=2
Sep 10 23:49:56.150227 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:56.149994    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:49:56.151582 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:49:56.151552    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7850-4bafd0\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="c1f7f048-5029-4763-8fbb-b33258181953" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:58.333808 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:58.333783    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:58.472783 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:58.472704    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj4dc\" (UniqueName: \"kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc\") pod \"c1f7f048-5029-4763-8fbb-b33258181953\" (UID: \"c1f7f048-5029-4763-8fbb-b33258181953\") "
Sep 10 23:49:58.474684 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:58.474661    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc" (OuterVolumeSpecName: "kube-api-access-vj4dc") pod "c1f7f048-5029-4763-8fbb-b33258181953" (UID: "c1f7f048-5029-4763-8fbb-b33258181953"). InnerVolumeSpecName "kube-api-access-vj4dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:49:58.573804 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:58.573768    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj4dc\" (UniqueName: \"kubernetes.io/projected/c1f7f048-5029-4763-8fbb-b33258181953-kube-api-access-vj4dc\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:49:59.166400 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:59.166368    2575 generic.go:356] "Generic (PLEG): container finished" podID="c1f7f048-5029-4763-8fbb-b33258181953" containerID="f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319" exitCode=137
Sep 10 23:49:59.166679 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:59.166408    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:49:59.166679 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:59.166414    2575 scope.go:117] "RemoveContainer" containerID="f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319"
Sep 10 23:49:59.175267 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:59.175250    2575 scope.go:117] "RemoveContainer" containerID="f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319"
Sep 10 23:49:59.175617 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:49:59.175600    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319\": container with ID starting with f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319 not found: ID does not exist" containerID="f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319"
Sep 10 23:49:59.175678 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:49:59.175626    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319"} err="failed to get container status \"f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319\": rpc error: code = NotFound desc = could not find container \"f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319\": container with ID starting with f92e5ee578c35ca426a9acf6b41407c8383e0544ee21487b5b3bd4eafaa77319 not found: ID does not exist"
Sep 10 23:49:59.176674 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:49:59.176655    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7850-4bafd0\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="c1f7f048-5029-4763-8fbb-b33258181953" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:50:00.228577 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:00.228547    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f7f048-5029-4763-8fbb-b33258181953" path="/var/lib/kubelet/pods/c1f7f048-5029-4763-8fbb-b33258181953/volumes"
Sep 10 23:50:00.229091 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:00.229066    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7850-4bafd0\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="c1f7f048-5029-4763-8fbb-b33258181953" pod="opendatahub/test-curl-7850-4bafd0"
Sep 10 23:50:01.409437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.409397    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:01.410148 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.410122    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f7f048-5029-4763-8fbb-b33258181953" containerName="test-curl-7850-4bafd0"
Sep 10 23:50:01.410148 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.410149    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f7f048-5029-4763-8fbb-b33258181953" containerName="test-curl-7850-4bafd0"
Sep 10 23:50:01.410352 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.410248    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f7f048-5029-4763-8fbb-b33258181953" containerName="test-curl-7850-4bafd0"
Sep 10 23:50:01.416382 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.416358    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.418643 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.418617    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-643c297d-serving-cert\""
Sep 10 23:50:01.426943 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.426920    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:01.503940 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.503904    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.504145 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.503953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.504145 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.503977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9szh\" (UniqueName: \"kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.504145 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.504009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.605396 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.605349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.605613 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.605415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.605613 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.605458    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9szh\" (UniqueName: \"kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.605613 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.605532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.605613 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:01.605536    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-serving-cert: secret "maas-api-e2e-ait-643c297d-serving-cert" not found
Sep 10 23:50:01.605613 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:01.605607    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls podName:1cc7f795-ffdd-4130-b7c2-9f680f9f97de nodeName:}" failed. No retries permitted until 2026-09-10 23:50:02.105586033 +0000 UTC m=+792.371582899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls") pod "maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de") : secret "maas-api-e2e-ait-643c297d-serving-cert" not found
Sep 10 23:50:01.606172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.606151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.607909 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.607881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:01.613782 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:01.613763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9szh\" (UniqueName: \"kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:02.110881 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:02.110847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:02.111039 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:02.110994    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-serving-cert: secret "maas-api-e2e-ait-643c297d-serving-cert" not found
Sep 10 23:50:02.111088 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:02.111066    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls podName:1cc7f795-ffdd-4130-b7c2-9f680f9f97de nodeName:}" failed. No retries permitted until 2026-09-10 23:50:03.111050658 +0000 UTC m=+793.377047511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls") pod "maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de") : secret "maas-api-e2e-ait-643c297d-serving-cert" not found
Sep 10 23:50:02.160541 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:02.160517    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:50:03.122357 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:03.122326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:03.124831 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:03.124811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:03.228582 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:03.228550    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:03.502407 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:03.502381    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:03.504376 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:50:03.504351    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc7f795_ffdd_4130_b7c2_9f680f9f97de.slice/crio-749f045fb4af41f4debc11ea01d297a4480aade400aa4e704e6308a11a5cb292 WatchSource:0}: Error finding container 749f045fb4af41f4debc11ea01d297a4480aade400aa4e704e6308a11a5cb292: Status 404 returned error can't find the container with id 749f045fb4af41f4debc11ea01d297a4480aade400aa4e704e6308a11a5cb292
Sep 10 23:50:04.195018 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:04.194936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" event={"ID":"1cc7f795-ffdd-4130-b7c2-9f680f9f97de","Type":"ContainerStarted","Data":"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"}
Sep 10 23:50:04.195018 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:04.194986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" event={"ID":"1cc7f795-ffdd-4130-b7c2-9f680f9f97de","Type":"ContainerStarted","Data":"749f045fb4af41f4debc11ea01d297a4480aade400aa4e704e6308a11a5cb292"}
Sep 10 23:50:04.195458 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:04.195068    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:04.218633 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:04.218579    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" podStartSLOduration=2.937646734 podStartE2EDuration="3.218565089s" totalImagesPullingTime="280.918355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:04.216132042 +0000 UTC m=+794.482128935" watchObservedRunningTime="2026-09-10 23:50:04.218565089 +0000 UTC m=+794.484561994"
Sep 10 23:50:05.006080 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.006044    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:50:05.010009 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.009986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.015275 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.015249    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-66836f-serving-cert\""
Sep 10 23:50:05.029797 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.029774    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:50:05.141907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.141870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.141907 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.141912    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.142115 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.141979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.142115 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.142047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89w4f\" (UniqueName: \"kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.243203 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.243174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89w4f\" (UniqueName: \"kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.243627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.243232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.243627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.243257    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.243627 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.243291    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.244054 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.244034    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.246049 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.246025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.246184 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.246055    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.257928 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.257865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89w4f\" (UniqueName: \"kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f\") pod \"maas-api-e2e-autores-66836f-cdbc5476f-6gd7m\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.321127 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.321094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:05.606168 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.606136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:50:05.609668 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:50:05.609636    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59423c6_cc67_4364_a038_a1c4fe0ebefb.slice/crio-f4371c93f53b73b1a53037578aacf78524b9a023516ed92818139f163aafa2bd WatchSource:0}: Error finding container f4371c93f53b73b1a53037578aacf78524b9a023516ed92818139f163aafa2bd: Status 404 returned error can't find the container with id f4371c93f53b73b1a53037578aacf78524b9a023516ed92818139f163aafa2bd
Sep 10 23:50:05.954105 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.954032    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:50:05.958994 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.958974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:05.962353 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.962313    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-66836f\""
Sep 10 23:50:05.962512 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.962416    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-66836f-dockercfg-gv2bn\""
Sep 10 23:50:05.972896 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:05.972875    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:50:06.051635 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.051613    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmjfj\" (UniqueName: \"kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.051767 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.051749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.152718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.152690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.152891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.152763    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmjfj\" (UniqueName: \"kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.153410 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.153386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.163718 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.163689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmjfj\" (UniqueName: \"kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj\") pod \"payload-processing-e2e-autores-66836f-5bbddd787-4f78t\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") " pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.207305 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.207243    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" event={"ID":"d59423c6-cc67-4364-a038-a1c4fe0ebefb","Type":"ContainerStarted","Data":"27c6108d94feb2bd6fb5de117f2ff055129b8f882ebd8a3968dcf68c94d26740"}
Sep 10 23:50:06.207305 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.207297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" event={"ID":"d59423c6-cc67-4364-a038-a1c4fe0ebefb","Type":"ContainerStarted","Data":"f4371c93f53b73b1a53037578aacf78524b9a023516ed92818139f163aafa2bd"}
Sep 10 23:50:06.207441 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.207377    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:06.229823 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.229780    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" podStartSLOduration=1.82519029 podStartE2EDuration="2.22976369s" totalImagesPullingTime="404.5734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:06.22779534 +0000 UTC m=+796.493792212" watchObservedRunningTime="2026-09-10 23:50:06.22976369 +0000 UTC m=+796.495760558"
Sep 10 23:50:06.270386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.270354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:06.569317 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:06.569095    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:50:06.571967 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:50:06.571934    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386d893d_60ab_43b6_b0ee_35427fb89d58.slice/crio-8afc75ccac84c550ceebcaa9aed6ba58ccc5df488fa6e844b62bea4a94a7c23b WatchSource:0}: Error finding container 8afc75ccac84c550ceebcaa9aed6ba58ccc5df488fa6e844b62bea4a94a7c23b: Status 404 returned error can't find the container with id 8afc75ccac84c550ceebcaa9aed6ba58ccc5df488fa6e844b62bea4a94a7c23b
Sep 10 23:50:07.214881 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:07.214851    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" event={"ID":"386d893d-60ab-43b6-b0ee-35427fb89d58","Type":"ContainerStarted","Data":"4f60d9028a4f8edd5082db28d66f4bc97000bec9a8327672454653236f05ddc3"}
Sep 10 23:50:07.215055 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:07.214901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" event={"ID":"386d893d-60ab-43b6-b0ee-35427fb89d58","Type":"ContainerStarted","Data":"8afc75ccac84c550ceebcaa9aed6ba58ccc5df488fa6e844b62bea4a94a7c23b"}
Sep 10 23:50:07.215149 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:07.215124    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:07.234368 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:07.234321    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" podStartSLOduration=2.234304017 podStartE2EDuration="2.234304017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:07.232918103 +0000 UTC m=+797.498914993" watchObservedRunningTime="2026-09-10 23:50:07.234304017 +0000 UTC m=+797.500300887"
Sep 10 23:50:10.205050 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:10.205011    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:12.220370 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:12.220292    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:50:18.228064 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:18.228030    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:50:28.496860 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.496785    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:50:28.500880 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.500858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.503135 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.503113    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-13374d-serving-cert\""
Sep 10 23:50:28.511664 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.511624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:50:28.574318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.574284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzc2f\" (UniqueName: \"kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.574461 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.574401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.574461 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.574446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.574554 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.574476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.675887 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.675850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.675887 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.675895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.676123 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.675922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.676123 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.675941    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzc2f\" (UniqueName: \"kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.676123 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:28.676023    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-serving-cert: secret "maas-api-e2e-inf-b-13374d-serving-cert" not found
Sep 10 23:50:28.676282 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:28.676125    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls podName:2bce7cfe-c6f6-4291-a048-9a462a3729cb nodeName:}" failed. No retries permitted until 2026-09-10 23:50:29.176105816 +0000 UTC m=+819.442102670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls") pod "maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" (UID: "2bce7cfe-c6f6-4291-a048-9a462a3729cb") : secret "maas-api-e2e-inf-b-13374d-serving-cert" not found
Sep 10 23:50:28.676661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.676638    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.678463 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.678435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:28.686206 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:28.686187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzc2f\" (UniqueName: \"kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:29.181276 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:29.181235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:29.183818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:29.183788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-13374d-5c6677db67-khn9s\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:29.412995 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:29.412961    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:29.693005 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:29.692980    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:50:29.695097 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:50:29.695070    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bce7cfe_c6f6_4291_a048_9a462a3729cb.slice/crio-d6c7d18c784bdae3c55bb2db97b15cd27d9651f2b5e7f7e657e06b65b16c6279 WatchSource:0}: Error finding container d6c7d18c784bdae3c55bb2db97b15cd27d9651f2b5e7f7e657e06b65b16c6279: Status 404 returned error can't find the container with id d6c7d18c784bdae3c55bb2db97b15cd27d9651f2b5e7f7e657e06b65b16c6279
Sep 10 23:50:30.332600 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:30.332568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" event={"ID":"2bce7cfe-c6f6-4291-a048-9a462a3729cb","Type":"ContainerStarted","Data":"d6c7d18c784bdae3c55bb2db97b15cd27d9651f2b5e7f7e657e06b65b16c6279"}
Sep 10 23:50:31.339468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:31.339432    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" event={"ID":"2bce7cfe-c6f6-4291-a048-9a462a3729cb","Type":"ContainerStarted","Data":"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"}
Sep 10 23:50:31.339867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:31.339519    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:31.358960 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:31.358914    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" podStartSLOduration=2.715176512 podStartE2EDuration="3.358899647s" totalImagesPullingTime="643.723135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:31.355938731 +0000 UTC m=+821.621935627" watchObservedRunningTime="2026-09-10 23:50:31.358899647 +0000 UTC m=+821.624896517"
Sep 10 23:50:37.349785 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:37.349752    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:50:43.000330 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.000296    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:43.000811 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.000539    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" podUID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" containerName="maas-api" containerID="cri-o://d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd" gracePeriod=30
Sep 10 23:50:43.258027 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.257965    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:43.400951 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.400910    2575 generic.go:356] "Generic (PLEG): container finished" podID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" containerID="d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd" exitCode=0
Sep 10 23:50:43.400951 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.400953    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"
Sep 10 23:50:43.401348 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.400990    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" event={"ID":"1cc7f795-ffdd-4130-b7c2-9f680f9f97de","Type":"ContainerDied","Data":"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"}
Sep 10 23:50:43.401348 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.401046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp" event={"ID":"1cc7f795-ffdd-4130-b7c2-9f680f9f97de","Type":"ContainerDied","Data":"749f045fb4af41f4debc11ea01d297a4480aade400aa4e704e6308a11a5cb292"}
Sep 10 23:50:43.401348 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.401076    2575 scope.go:117] "RemoveContainer" containerID="d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"
Sep 10 23:50:43.410863 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.410844    2575 scope.go:117] "RemoveContainer" containerID="d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"
Sep 10 23:50:43.410943 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.410889    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca\") pod \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") "
Sep 10 23:50:43.411006 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.410989    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9szh\" (UniqueName: \"kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh\") pod \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") "
Sep 10 23:50:43.411086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.411068    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") pod \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") "
Sep 10 23:50:43.411713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.411216    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs\") pod \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\" (UID: \"1cc7f795-ffdd-4130-b7c2-9f680f9f97de\") "
Sep 10 23:50:43.411713 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:43.411237    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd\": container with ID starting with d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd not found: ID does not exist" containerID="d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"
Sep 10 23:50:43.411713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.411271    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd"} err="failed to get container status \"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd\": rpc error: code = NotFound desc = could not find container \"d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd\": container with ID starting with d703647a69f15407839906216eac318177759fa2a75b52ae7635c8f6a86fa0fd not found: ID does not exist"
Sep 10 23:50:43.411713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.411277    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cc7f795-ffdd-4130-b7c2-9f680f9f97de" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:43.411713 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.411676    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:50:43.413234 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.413206    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cc7f795-ffdd-4130-b7c2-9f680f9f97de" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:50:43.413383 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.413364    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1cc7f795-ffdd-4130-b7c2-9f680f9f97de" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:50:43.413700 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.413679    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh" (OuterVolumeSpecName: "kube-api-access-r9szh") pod "1cc7f795-ffdd-4130-b7c2-9f680f9f97de" (UID: "1cc7f795-ffdd-4130-b7c2-9f680f9f97de"). InnerVolumeSpecName "kube-api-access-r9szh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:43.512856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.512786    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9szh\" (UniqueName: \"kubernetes.io/projected/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-kube-api-access-r9szh\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:50:43.512856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.512815    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:50:43.512856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.512827    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cc7f795-ffdd-4130-b7c2-9f680f9f97de-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:50:43.723597 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.723564    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:43.730046 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:43.730024    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-643c297d-7b4cd86c89-vfkwp"]
Sep 10 23:50:44.228009 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:44.227974    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" path="/var/lib/kubelet/pods/1cc7f795-ffdd-4130-b7c2-9f680f9f97de/volumes"
Sep 10 23:50:47.707737 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.707697    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:50:47.708214 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.708198    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" containerName="maas-api"
Sep 10 23:50:47.708256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.708217    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" containerName="maas-api"
Sep 10 23:50:47.708290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.708284    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc7f795-ffdd-4130-b7c2-9f680f9f97de" containerName="maas-api"
Sep 10 23:50:47.711436 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.711419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.715149 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:47.715097    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.715679 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:47.715653    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-migrate-58451477-serving-cert\" is forbidden: User \"system:node:ip-10-0-134-115.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-134-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-58451477-serving-cert\"" type="*v1.Secret"
Sep 10 23:50:47.746992 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.746959    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:50:47.856866 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.856829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.857071 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.856879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.857071 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.856901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.857071 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.857022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz7mz\" (UniqueName: \"kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.958058 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.957968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.958058 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.958014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.958058 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.958035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.958281 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.958075    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz7mz\" (UniqueName: \"kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.958705 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.958682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.960656 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.960632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:47.966096 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:47.966073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz7mz\" (UniqueName: \"kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:48.959136 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:48.959103    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-serving-cert: failed to sync secret cache: timed out waiting for the condition
Sep 10 23:50:48.959626 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:50:48.959200    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls podName:6ffaa815-af47-4aab-8000-ec8dc74b5bec nodeName:}" failed. No retries permitted until 2026-09-10 23:50:49.45917563 +0000 UTC m=+839.725172500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls") pod "maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" (UID: "6ffaa815-af47-4aab-8000-ec8dc74b5bec") : failed to sync secret cache: timed out waiting for the condition
Sep 10 23:50:49.238804 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:49.238723    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-58451477-serving-cert\""
Sep 10 23:50:49.472686 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:49.472648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:49.475207 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:49.475183    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") pod \"maas-api-e2e-migrate-58451477-59c88ffc87-jdffl\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:49.522074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:49.522009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:49.798313 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:49.798129    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:50:49.800976 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:50:49.800950    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffaa815_af47_4aab_8000_ec8dc74b5bec.slice/crio-f6169977d0b145e6e2cda4e6f24e0abb439558edced3fa2a6aeeb60dd79fa39a WatchSource:0}: Error finding container f6169977d0b145e6e2cda4e6f24e0abb439558edced3fa2a6aeeb60dd79fa39a: Status 404 returned error can't find the container with id f6169977d0b145e6e2cda4e6f24e0abb439558edced3fa2a6aeeb60dd79fa39a
Sep 10 23:50:50.440336 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:50.440305    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" event={"ID":"6ffaa815-af47-4aab-8000-ec8dc74b5bec","Type":"ContainerStarted","Data":"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"}
Sep 10 23:50:50.440734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:50.440359    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" event={"ID":"6ffaa815-af47-4aab-8000-ec8dc74b5bec","Type":"ContainerStarted","Data":"f6169977d0b145e6e2cda4e6f24e0abb439558edced3fa2a6aeeb60dd79fa39a"}
Sep 10 23:50:50.440734 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:50.440444    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:50:50.458858 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:50.458762    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" podStartSLOduration=3.000327344 podStartE2EDuration="3.458746181s" totalImagesPullingTime="458.418837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:50.456656627 +0000 UTC m=+840.722653516" watchObservedRunningTime="2026-09-10 23:50:50.458746181 +0000 UTC m=+840.724743051"
Sep 10 23:50:56.452112 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:50:56.452079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:51:39.658279 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:39.658245    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:51:39.659055 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:39.659001    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" containerName="maas-api" containerID="cri-o://9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed" gracePeriod=30
Sep 10 23:51:39.915688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:39.915630    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:51:40.055655 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.055628    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") pod \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") "
Sep 10 23:51:40.055839 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.055707    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca\") pod \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") "
Sep 10 23:51:40.055839 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.055816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz7mz\" (UniqueName: \"kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz\") pod \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") "
Sep 10 23:51:40.055965 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.055844    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs\") pod \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\" (UID: \"6ffaa815-af47-4aab-8000-ec8dc74b5bec\") "
Sep 10 23:51:40.056150 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.056114    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ffaa815-af47-4aab-8000-ec8dc74b5bec" (UID: "6ffaa815-af47-4aab-8000-ec8dc74b5bec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:51:40.056290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.056261    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffaa815-af47-4aab-8000-ec8dc74b5bec-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:51:40.057855 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.057828    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ffaa815-af47-4aab-8000-ec8dc74b5bec" (UID: "6ffaa815-af47-4aab-8000-ec8dc74b5bec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:51:40.058029 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.057951    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz" (OuterVolumeSpecName: "kube-api-access-hz7mz") pod "6ffaa815-af47-4aab-8000-ec8dc74b5bec" (UID: "6ffaa815-af47-4aab-8000-ec8dc74b5bec"). InnerVolumeSpecName "kube-api-access-hz7mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:51:40.058091 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.058059    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6ffaa815-af47-4aab-8000-ec8dc74b5bec" (UID: "6ffaa815-af47-4aab-8000-ec8dc74b5bec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:51:40.157729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.157699    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:51:40.157729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.157721    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz7mz\" (UniqueName: \"kubernetes.io/projected/6ffaa815-af47-4aab-8000-ec8dc74b5bec-kube-api-access-hz7mz\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:51:40.157729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.157731    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ffaa815-af47-4aab-8000-ec8dc74b5bec-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:51:40.704859 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.704826    2575 generic.go:356] "Generic (PLEG): container finished" podID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" containerID="9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed" exitCode=0
Sep 10 23:51:40.705271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.704898    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"
Sep 10 23:51:40.705271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.704897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" event={"ID":"6ffaa815-af47-4aab-8000-ec8dc74b5bec","Type":"ContainerDied","Data":"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"}
Sep 10 23:51:40.705271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.705026    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl" event={"ID":"6ffaa815-af47-4aab-8000-ec8dc74b5bec","Type":"ContainerDied","Data":"f6169977d0b145e6e2cda4e6f24e0abb439558edced3fa2a6aeeb60dd79fa39a"}
Sep 10 23:51:40.705271 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.705050    2575 scope.go:117] "RemoveContainer" containerID="9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"
Sep 10 23:51:40.714185 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.714169    2575 scope.go:117] "RemoveContainer" containerID="9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"
Sep 10 23:51:40.714464 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:51:40.714446    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed\": container with ID starting with 9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed not found: ID does not exist" containerID="9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"
Sep 10 23:51:40.714538 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.714474    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed"} err="failed to get container status \"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed\": rpc error: code = NotFound desc = could not find container \"9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed\": container with ID starting with 9dd79547f13853db70e16e1aba4acebb2965ed575bb6632b52aadd5818ca18ed not found: ID does not exist"
Sep 10 23:51:40.730378 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.730351    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:51:40.734986 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:40.734962    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58451477-59c88ffc87-jdffl"]
Sep 10 23:51:42.227901 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:42.227798    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" path="/var/lib/kubelet/pods/6ffaa815-af47-4aab-8000-ec8dc74b5bec/volumes"
Sep 10 23:51:44.513891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.513856    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:51:44.514334 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.514321    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" containerName="maas-api"
Sep 10 23:51:44.514334 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.514335    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" containerName="maas-api"
Sep 10 23:51:44.514429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.514420    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ffaa815-af47-4aab-8000-ec8dc74b5bec" containerName="maas-api"
Sep 10 23:51:44.518979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.518963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.521346 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.521319    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f9a25dcc-serving-cert\""
Sep 10 23:51:44.529321 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.529295    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:51:44.603090 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.603058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.603245 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.603105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.603245 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.603221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtkf7\" (UniqueName: \"kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.603313 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.603276    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.704139 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.704108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.704318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.704187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtkf7\" (UniqueName: \"kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.704318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.704222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.704318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.704263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.704318 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:51:44.704262    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-serving-cert: secret "maas-api-e2e-ait-f9a25dcc-serving-cert" not found
Sep 10 23:51:44.704478 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:51:44.704368    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls podName:c4942b37-4521-486e-9682-6d507e3e4330 nodeName:}" failed. No retries permitted until 2026-09-10 23:51:45.204345714 +0000 UTC m=+895.470342571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls") pod "maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" (UID: "c4942b37-4521-486e-9682-6d507e3e4330") : secret "maas-api-e2e-ait-f9a25dcc-serving-cert" not found
Sep 10 23:51:44.704921 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.704899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.706611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.706595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:44.729202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:44.729179    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtkf7\" (UniqueName: \"kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:45.210086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:45.210043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:45.212571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:45.212546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") pod \"maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:45.429678 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:45.429646    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:45.703842 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:45.703811    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:51:45.734096 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:45.734039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" event={"ID":"c4942b37-4521-486e-9682-6d507e3e4330","Type":"ContainerStarted","Data":"2ac7226e4d3ec286b5216de876dd26abb19435366622ada1a8a0a46f6dbd17a3"}
Sep 10 23:51:46.741892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:46.741858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" event={"ID":"c4942b37-4521-486e-9682-6d507e3e4330","Type":"ContainerStarted","Data":"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"}
Sep 10 23:51:46.742276 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:46.741926    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:51:46.760768 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:46.760722    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" podStartSLOduration=2.216207284 podStartE2EDuration="2.760707842s" totalImagesPullingTime="544.500558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:46.758955219 +0000 UTC m=+897.024952105" watchObservedRunningTime="2026-09-10 23:51:46.760707842 +0000 UTC m=+897.026704712"
Sep 10 23:51:52.753609 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:51:52.753573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:52:06.625068 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.625036    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:52:06.625565 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.625267    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" podUID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" containerName="maas-api" containerID="cri-o://27c6108d94feb2bd6fb5de117f2ff055129b8f882ebd8a3968dcf68c94d26740" gracePeriod=30
Sep 10 23:52:06.690793 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.690752    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:52:06.691063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.691018    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" podUID="386d893d-60ab-43b6-b0ee-35427fb89d58" containerName="payload-processing" containerID="cri-o://4f60d9028a4f8edd5082db28d66f4bc97000bec9a8327672454653236f05ddc3" gracePeriod=30
Sep 10 23:52:06.847845 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.847809    2575 generic.go:356] "Generic (PLEG): container finished" podID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" containerID="27c6108d94feb2bd6fb5de117f2ff055129b8f882ebd8a3968dcf68c94d26740" exitCode=0
Sep 10 23:52:06.848014 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.847891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" event={"ID":"d59423c6-cc67-4364-a038-a1c4fe0ebefb","Type":"ContainerDied","Data":"27c6108d94feb2bd6fb5de117f2ff055129b8f882ebd8a3968dcf68c94d26740"}
Sep 10 23:52:06.849700 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.849673    2575 generic.go:356] "Generic (PLEG): container finished" podID="386d893d-60ab-43b6-b0ee-35427fb89d58" containerID="4f60d9028a4f8edd5082db28d66f4bc97000bec9a8327672454653236f05ddc3" exitCode=0
Sep 10 23:52:06.849838 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.849725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" event={"ID":"386d893d-60ab-43b6-b0ee-35427fb89d58","Type":"ContainerDied","Data":"4f60d9028a4f8edd5082db28d66f4bc97000bec9a8327672454653236f05ddc3"}
Sep 10 23:52:06.949514 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.949468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:52:06.982925 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:06.982900    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:52:07.019464 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019440    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca\") pod \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") "
Sep 10 23:52:07.019639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019482    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89w4f\" (UniqueName: \"kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f\") pod \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") "
Sep 10 23:52:07.019639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019543    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume\") pod \"386d893d-60ab-43b6-b0ee-35427fb89d58\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") "
Sep 10 23:52:07.019639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019569    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmjfj\" (UniqueName: \"kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj\") pod \"386d893d-60ab-43b6-b0ee-35427fb89d58\" (UID: \"386d893d-60ab-43b6-b0ee-35427fb89d58\") "
Sep 10 23:52:07.019639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019598    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls\") pod \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") "
Sep 10 23:52:07.019639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019638    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs\") pod \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\" (UID: \"d59423c6-cc67-4364-a038-a1c4fe0ebefb\") "
Sep 10 23:52:07.019908 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019831    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d59423c6-cc67-4364-a038-a1c4fe0ebefb" (UID: "d59423c6-cc67-4364-a038-a1c4fe0ebefb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:52:07.020022 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.019981    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "386d893d-60ab-43b6-b0ee-35427fb89d58" (UID: "386d893d-60ab-43b6-b0ee-35427fb89d58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:52:07.020084 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.020028    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59423c6-cc67-4364-a038-a1c4fe0ebefb-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.021891 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.021865    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d59423c6-cc67-4364-a038-a1c4fe0ebefb" (UID: "d59423c6-cc67-4364-a038-a1c4fe0ebefb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:07.022278 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.022250    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f" (OuterVolumeSpecName: "kube-api-access-89w4f") pod "d59423c6-cc67-4364-a038-a1c4fe0ebefb" (UID: "d59423c6-cc67-4364-a038-a1c4fe0ebefb"). InnerVolumeSpecName "kube-api-access-89w4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:52:07.022377 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.022320    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d59423c6-cc67-4364-a038-a1c4fe0ebefb" (UID: "d59423c6-cc67-4364-a038-a1c4fe0ebefb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:07.022423 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.022385    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj" (OuterVolumeSpecName: "kube-api-access-wmjfj") pod "386d893d-60ab-43b6-b0ee-35427fb89d58" (UID: "386d893d-60ab-43b6-b0ee-35427fb89d58"). InnerVolumeSpecName "kube-api-access-wmjfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:52:07.120876 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.120849    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.120876 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.120870    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d59423c6-cc67-4364-a038-a1c4fe0ebefb-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.120876 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.120879    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89w4f\" (UniqueName: \"kubernetes.io/projected/d59423c6-cc67-4364-a038-a1c4fe0ebefb-kube-api-access-89w4f\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.121075 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.120888    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/386d893d-60ab-43b6-b0ee-35427fb89d58-plugins-config-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.121075 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.120896    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmjfj\" (UniqueName: \"kubernetes.io/projected/386d893d-60ab-43b6-b0ee-35427fb89d58-kube-api-access-wmjfj\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:07.856320 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.856290    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"
Sep 10 23:52:07.856801 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.856329    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t" event={"ID":"386d893d-60ab-43b6-b0ee-35427fb89d58","Type":"ContainerDied","Data":"8afc75ccac84c550ceebcaa9aed6ba58ccc5df488fa6e844b62bea4a94a7c23b"}
Sep 10 23:52:07.856801 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.856388    2575 scope.go:117] "RemoveContainer" containerID="4f60d9028a4f8edd5082db28d66f4bc97000bec9a8327672454653236f05ddc3"
Sep 10 23:52:07.857968 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.857945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m" event={"ID":"d59423c6-cc67-4364-a038-a1c4fe0ebefb","Type":"ContainerDied","Data":"f4371c93f53b73b1a53037578aacf78524b9a023516ed92818139f163aafa2bd"}
Sep 10 23:52:07.858077 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.857975    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"
Sep 10 23:52:07.865997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.865978    2575 scope.go:117] "RemoveContainer" containerID="27c6108d94feb2bd6fb5de117f2ff055129b8f882ebd8a3968dcf68c94d26740"
Sep 10 23:52:07.881033 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.881013    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:52:07.884771 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.884751    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-66836f-5bbddd787-4f78t"]
Sep 10 23:52:07.896242 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.896182    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:52:07.903523 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:07.903502    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-66836f-cdbc5476f-6gd7m"]
Sep 10 23:52:08.228313 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:08.228237    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="386d893d-60ab-43b6-b0ee-35427fb89d58" path="/var/lib/kubelet/pods/386d893d-60ab-43b6-b0ee-35427fb89d58/volumes"
Sep 10 23:52:08.228657 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:08.228643    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" path="/var/lib/kubelet/pods/d59423c6-cc67-4364-a038-a1c4fe0ebefb/volumes"
Sep 10 23:52:14.957802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:14.957753    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:52:14.958343 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:14.958029    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" podUID="c4942b37-4521-486e-9682-6d507e3e4330" containerName="maas-api" containerID="cri-o://1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd" gracePeriod=30
Sep 10 23:52:15.215817 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.215760    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:52:15.295509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.295462    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca\") pod \"c4942b37-4521-486e-9682-6d507e3e4330\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") "
Sep 10 23:52:15.295668 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.295528    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") pod \"c4942b37-4521-486e-9682-6d507e3e4330\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") "
Sep 10 23:52:15.295715 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.295674    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtkf7\" (UniqueName: \"kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7\") pod \"c4942b37-4521-486e-9682-6d507e3e4330\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") "
Sep 10 23:52:15.295750 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.295716    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs\") pod \"c4942b37-4521-486e-9682-6d507e3e4330\" (UID: \"c4942b37-4521-486e-9682-6d507e3e4330\") "
Sep 10 23:52:15.295887 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.295858    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4942b37-4521-486e-9682-6d507e3e4330" (UID: "c4942b37-4521-486e-9682-6d507e3e4330"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:52:15.296234 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.296205    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4942b37-4521-486e-9682-6d507e3e4330-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:15.297847 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.297823    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7" (OuterVolumeSpecName: "kube-api-access-qtkf7") pod "c4942b37-4521-486e-9682-6d507e3e4330" (UID: "c4942b37-4521-486e-9682-6d507e3e4330"). InnerVolumeSpecName "kube-api-access-qtkf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:52:15.298290 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.298271    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4942b37-4521-486e-9682-6d507e3e4330" (UID: "c4942b37-4521-486e-9682-6d507e3e4330"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:15.298355 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.298292    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c4942b37-4521-486e-9682-6d507e3e4330" (UID: "c4942b37-4521-486e-9682-6d507e3e4330"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:15.396825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.396795    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtkf7\" (UniqueName: \"kubernetes.io/projected/c4942b37-4521-486e-9682-6d507e3e4330-kube-api-access-qtkf7\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:15.396825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.396825    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:15.396992 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.396838    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4942b37-4521-486e-9682-6d507e3e4330-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:15.902573 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.902535    2575 generic.go:356] "Generic (PLEG): container finished" podID="c4942b37-4521-486e-9682-6d507e3e4330" containerID="1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd" exitCode=0
Sep 10 23:52:15.902747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.902586    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"
Sep 10 23:52:15.902747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.902612    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" event={"ID":"c4942b37-4521-486e-9682-6d507e3e4330","Type":"ContainerDied","Data":"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"}
Sep 10 23:52:15.902747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.902663    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68" event={"ID":"c4942b37-4521-486e-9682-6d507e3e4330","Type":"ContainerDied","Data":"2ac7226e4d3ec286b5216de876dd26abb19435366622ada1a8a0a46f6dbd17a3"}
Sep 10 23:52:15.902747 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.902683    2575 scope.go:117] "RemoveContainer" containerID="1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"
Sep 10 23:52:15.912326 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.912305    2575 scope.go:117] "RemoveContainer" containerID="1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"
Sep 10 23:52:15.912639 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:52:15.912620    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd\": container with ID starting with 1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd not found: ID does not exist" containerID="1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"
Sep 10 23:52:15.912723 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.912646    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd"} err="failed to get container status \"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd\": rpc error: code = NotFound desc = could not find container \"1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd\": container with ID starting with 1dd7e66b2289aae824ead68ae0a5bf381eaddc7f3e43368527f160074c0cdffd not found: ID does not exist"
Sep 10 23:52:15.934696 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.934670    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:52:15.941237 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:15.941217    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9a25dcc-755b6f8f6d-n9x68"]
Sep 10 23:52:16.227536 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:16.227453    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4942b37-4521-486e-9682-6d507e3e4330" path="/var/lib/kubelet/pods/c4942b37-4521-486e-9682-6d507e3e4330/volumes"
Sep 10 23:52:18.693206 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693173    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:52:18.693653 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693638    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4942b37-4521-486e-9682-6d507e3e4330" containerName="maas-api"
Sep 10 23:52:18.693697 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693654    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4942b37-4521-486e-9682-6d507e3e4330" containerName="maas-api"
Sep 10 23:52:18.693697 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693674    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="386d893d-60ab-43b6-b0ee-35427fb89d58" containerName="payload-processing"
Sep 10 23:52:18.693697 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693679    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="386d893d-60ab-43b6-b0ee-35427fb89d58" containerName="payload-processing"
Sep 10 23:52:18.693697 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693693    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" containerName="maas-api"
Sep 10 23:52:18.693818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693699    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" containerName="maas-api"
Sep 10 23:52:18.693818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693762    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4942b37-4521-486e-9682-6d507e3e4330" containerName="maas-api"
Sep 10 23:52:18.693818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693771    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d59423c6-cc67-4364-a038-a1c4fe0ebefb" containerName="maas-api"
Sep 10 23:52:18.693818 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.693780    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="386d893d-60ab-43b6-b0ee-35427fb89d58" containerName="payload-processing"
Sep 10 23:52:18.698525 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.698480    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.700674 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.700649    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-677d3c-serving-cert\""
Sep 10 23:52:18.710427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.710404    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:52:18.827983 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.827952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wccqf\" (UniqueName: \"kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.828142 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.828004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.828142 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.828114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.828216 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.828149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.929261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.929232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.929373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.929269    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.929373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.929300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wccqf\" (UniqueName: \"kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.929373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.929340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.929526 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:52:18.929386    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-serving-cert: secret "maas-api-e2e-explicit-677d3c-serving-cert" not found
Sep 10 23:52:18.929526 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:52:18.929468    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls podName:bbd5bb05-a0e2-49b5-999f-a44a2ea532cc nodeName:}" failed. No retries permitted until 2026-09-10 23:52:19.42944724 +0000 UTC m=+929.695444091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls") pod "maas-api-e2e-explicit-677d3c-db64867-mjrg5" (UID: "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc") : secret "maas-api-e2e-explicit-677d3c-serving-cert" not found
Sep 10 23:52:18.929967 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.929944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.932040 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.932020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:18.943370 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:18.943316    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wccqf\" (UniqueName: \"kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:19.411925 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.411896    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:52:19.417460 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.417437    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.420030 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.420009    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-677d3c-dockercfg-pzdsw\""
Sep 10 23:52:19.420140 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.420065    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-677d3c\""
Sep 10 23:52:19.426139 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.426117    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:52:19.434757 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.434740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:19.437398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.437378    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") pod \"maas-api-e2e-explicit-677d3c-db64867-mjrg5\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:19.535676 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.535647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.535821 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.535702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtkw\" (UniqueName: \"kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.609542 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.609507    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:19.636778 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.636749    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.636964 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.636793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtkw\" (UniqueName: \"kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.637398 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.637375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.646011 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.645988    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtkw\" (UniqueName: \"kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw\") pod \"payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.729612 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.729585    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:19.891240 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.891215    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:52:19.893023 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:52:19.892998    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd5bb05_a0e2_49b5_999f_a44a2ea532cc.slice/crio-01d2c6a6ddbbb80728b8824b4ff4e7c2b38e51447abf4546c130f48fc6f8898a WatchSource:0}: Error finding container 01d2c6a6ddbbb80728b8824b4ff4e7c2b38e51447abf4546c130f48fc6f8898a: Status 404 returned error can't find the container with id 01d2c6a6ddbbb80728b8824b4ff4e7c2b38e51447abf4546c130f48fc6f8898a
Sep 10 23:52:19.932771 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:19.932702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" event={"ID":"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc","Type":"ContainerStarted","Data":"01d2c6a6ddbbb80728b8824b4ff4e7c2b38e51447abf4546c130f48fc6f8898a"}
Sep 10 23:52:20.015835 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.015813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:52:20.017353 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:52:20.017323    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a8ae6f_0fdb_4647_ae46_5b959ba3db9e.slice/crio-98092a9403a215610df58ba20297aacc535eefaaf9a0c77a9245c65c3e3d884d WatchSource:0}: Error finding container 98092a9403a215610df58ba20297aacc535eefaaf9a0c77a9245c65c3e3d884d: Status 404 returned error can't find the container with id 98092a9403a215610df58ba20297aacc535eefaaf9a0c77a9245c65c3e3d884d
Sep 10 23:52:20.939857 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.939818    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" event={"ID":"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e","Type":"ContainerStarted","Data":"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"}
Sep 10 23:52:20.940322 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.939869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" event={"ID":"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e","Type":"ContainerStarted","Data":"98092a9403a215610df58ba20297aacc535eefaaf9a0c77a9245c65c3e3d884d"}
Sep 10 23:52:20.940322 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.939921    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:20.941474 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.941449    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" event={"ID":"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc","Type":"ContainerStarted","Data":"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"}
Sep 10 23:52:20.941598 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.941541    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:20.962509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.962440    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" podStartSLOduration=1.962420921 podStartE2EDuration="1.962420921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:20.959579253 +0000 UTC m=+931.225576141" watchObservedRunningTime="2026-09-10 23:52:20.962420921 +0000 UTC m=+931.228417790"
Sep 10 23:52:20.982941 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:20.982897    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" podStartSLOduration=2.488548308 podStartE2EDuration="2.982879712s" totalImagesPullingTime="494.331404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:20.980858673 +0000 UTC m=+931.246855542" watchObservedRunningTime="2026-09-10 23:52:20.982879712 +0000 UTC m=+931.248876582"
Sep 10 23:52:21.420383 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.420354    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:21.423972 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.423948    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.426823 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.426799    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-15a3a094-serving-cert\""
Sep 10 23:52:21.443580 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.443476    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:21.555944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.555910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.556124 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.555977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vztm\" (UniqueName: \"kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.556214 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.556154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.556214 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.556194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.656735 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.656706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.656892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.656750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vztm\" (UniqueName: \"kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.656892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.656822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.656892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.656842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.657688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.657513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.659437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.659416    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.659437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.659427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.665520 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.665480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vztm\" (UniqueName: \"kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm\") pod \"maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:21.735725 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:21.735669    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:22.009138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.009112    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:22.010793 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:52:22.010766    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2875290f_2019_48de_a3f9_5fe2a880ee41.slice/crio-9a4036a6e873ab35cbc0d207e41bfea57302db8e3b0e4aeee78e59df0e467fcd WatchSource:0}: Error finding container 9a4036a6e873ab35cbc0d207e41bfea57302db8e3b0e4aeee78e59df0e467fcd: Status 404 returned error can't find the container with id 9a4036a6e873ab35cbc0d207e41bfea57302db8e3b0e4aeee78e59df0e467fcd
Sep 10 23:52:22.229074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.229044    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:22.233953 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.233898    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.236933 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.236911    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-15a3a094\""
Sep 10 23:52:22.237059 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.236980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-15a3a094-dockercfg-g4jp7\""
Sep 10 23:52:22.238561 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.238524    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:22.364871 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.364846    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.364990 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.364881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqf8g\" (UniqueName: \"kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.465744 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.465712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.465913 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.465750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqf8g\" (UniqueName: \"kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.466541 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.466521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.494445 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.494415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqf8g\" (UniqueName: \"kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g\") pod \"payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") " pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.548651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.548619    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.829855 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.829831    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:22.832099 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:52:22.832067    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a09bb65_dfdc_4a12_9c62_f6807e73996a.slice/crio-3357b5a9802a897fc3927c6642d63ee7991853f8fd0090f336f11d836dd09d88 WatchSource:0}: Error finding container 3357b5a9802a897fc3927c6642d63ee7991853f8fd0090f336f11d836dd09d88: Status 404 returned error can't find the container with id 3357b5a9802a897fc3927c6642d63ee7991853f8fd0090f336f11d836dd09d88
Sep 10 23:52:22.955095 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.955063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" event={"ID":"9a09bb65-dfdc-4a12-9c62-f6807e73996a","Type":"ContainerStarted","Data":"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"}
Sep 10 23:52:22.955268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.955125    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:22.955268 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.955152    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" event={"ID":"9a09bb65-dfdc-4a12-9c62-f6807e73996a","Type":"ContainerStarted","Data":"3357b5a9802a897fc3927c6642d63ee7991853f8fd0090f336f11d836dd09d88"}
Sep 10 23:52:22.956824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.956797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" event={"ID":"2875290f-2019-48de-a3f9-5fe2a880ee41","Type":"ContainerStarted","Data":"dc174b2066f2a8d534d0291b7efd8fdeab3575be25269b98864fddecd7cec597"}
Sep 10 23:52:22.956963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.956846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" event={"ID":"2875290f-2019-48de-a3f9-5fe2a880ee41","Type":"ContainerStarted","Data":"9a4036a6e873ab35cbc0d207e41bfea57302db8e3b0e4aeee78e59df0e467fcd"}
Sep 10 23:52:22.956963 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.956874    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:22.979378 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.979323    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" podStartSLOduration=0.979304082 podStartE2EDuration="979.304082ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:22.978401828 +0000 UTC m=+933.244398718" watchObservedRunningTime="2026-09-10 23:52:22.979304082 +0000 UTC m=+933.245300954"
Sep 10 23:52:22.998199 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:22.998151    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" podStartSLOduration=1.670825038 podStartE2EDuration="1.998133441s" totalImagesPullingTime="327.308403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:22.995934371 +0000 UTC m=+933.261931241" watchObservedRunningTime="2026-09-10 23:52:22.998133441 +0000 UTC m=+933.264130310"
Sep 10 23:52:26.953181 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:26.953153    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:52:28.968026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:28.968001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:31.950345 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:31.950313    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:52:33.965109 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:33.965070    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:51.999334 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:51.999299    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:51.999824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:51.999529    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" podUID="2875290f-2019-48de-a3f9-5fe2a880ee41" containerName="maas-api" containerID="cri-o://dc174b2066f2a8d534d0291b7efd8fdeab3575be25269b98864fddecd7cec597" gracePeriod=30
Sep 10 23:52:52.060026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.059997    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:52.060427 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.060226    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" podUID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" containerName="payload-processing" containerID="cri-o://8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa" gracePeriod=30
Sep 10 23:52:52.115807 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.115768    2575 generic.go:356] "Generic (PLEG): container finished" podID="2875290f-2019-48de-a3f9-5fe2a880ee41" containerID="dc174b2066f2a8d534d0291b7efd8fdeab3575be25269b98864fddecd7cec597" exitCode=0
Sep 10 23:52:52.116007 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.115844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" event={"ID":"2875290f-2019-48de-a3f9-5fe2a880ee41","Type":"ContainerDied","Data":"dc174b2066f2a8d534d0291b7efd8fdeab3575be25269b98864fddecd7cec597"}
Sep 10 23:52:52.323948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.323903    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:52.343327 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.343096    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:52.354722 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.354699    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs\") pod \"2875290f-2019-48de-a3f9-5fe2a880ee41\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") "
Sep 10 23:52:52.354824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.354759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls\") pod \"2875290f-2019-48de-a3f9-5fe2a880ee41\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") "
Sep 10 23:52:52.354824 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.354814    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vztm\" (UniqueName: \"kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm\") pod \"2875290f-2019-48de-a3f9-5fe2a880ee41\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") "
Sep 10 23:52:52.354919 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.354878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca\") pod \"2875290f-2019-48de-a3f9-5fe2a880ee41\" (UID: \"2875290f-2019-48de-a3f9-5fe2a880ee41\") "
Sep 10 23:52:52.355285 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.355250    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2875290f-2019-48de-a3f9-5fe2a880ee41" (UID: "2875290f-2019-48de-a3f9-5fe2a880ee41"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:52:52.356898 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.356874    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2875290f-2019-48de-a3f9-5fe2a880ee41" (UID: "2875290f-2019-48de-a3f9-5fe2a880ee41"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:52.357001 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.356984    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm" (OuterVolumeSpecName: "kube-api-access-2vztm") pod "2875290f-2019-48de-a3f9-5fe2a880ee41" (UID: "2875290f-2019-48de-a3f9-5fe2a880ee41"). InnerVolumeSpecName "kube-api-access-2vztm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:52:52.357401 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.357381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2875290f-2019-48de-a3f9-5fe2a880ee41" (UID: "2875290f-2019-48de-a3f9-5fe2a880ee41"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:52:52.455706 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.455676    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqf8g\" (UniqueName: \"kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g\") pod \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") "
Sep 10 23:52:52.455706 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.455712    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume\") pod \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\" (UID: \"9a09bb65-dfdc-4a12-9c62-f6807e73996a\") "
Sep 10 23:52:52.456016 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.456003    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:52.456016 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.456017    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2875290f-2019-48de-a3f9-5fe2a880ee41-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:52.456138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.456026    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vztm\" (UniqueName: \"kubernetes.io/projected/2875290f-2019-48de-a3f9-5fe2a880ee41-kube-api-access-2vztm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:52.456138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.456037    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2875290f-2019-48de-a3f9-5fe2a880ee41-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:52.456138 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.456102    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a09bb65-dfdc-4a12-9c62-f6807e73996a" (UID: "9a09bb65-dfdc-4a12-9c62-f6807e73996a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:52:52.457829 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.457806    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g" (OuterVolumeSpecName: "kube-api-access-dqf8g") pod "9a09bb65-dfdc-4a12-9c62-f6807e73996a" (UID: "9a09bb65-dfdc-4a12-9c62-f6807e73996a"). InnerVolumeSpecName "kube-api-access-dqf8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:52:52.557531 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.557503    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqf8g\" (UniqueName: \"kubernetes.io/projected/9a09bb65-dfdc-4a12-9c62-f6807e73996a-kube-api-access-dqf8g\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:52.557531 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:52.557525    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a09bb65-dfdc-4a12-9c62-f6807e73996a-plugins-config-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:52:53.123269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.123231    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" containerID="8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa" exitCode=0
Sep 10 23:52:53.123754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.123296    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"
Sep 10 23:52:53.123754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.123323    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" event={"ID":"9a09bb65-dfdc-4a12-9c62-f6807e73996a","Type":"ContainerDied","Data":"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"}
Sep 10 23:52:53.123754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.123379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv" event={"ID":"9a09bb65-dfdc-4a12-9c62-f6807e73996a","Type":"ContainerDied","Data":"3357b5a9802a897fc3927c6642d63ee7991853f8fd0090f336f11d836dd09d88"}
Sep 10 23:52:53.123754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.123406    2575 scope.go:117] "RemoveContainer" containerID="8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"
Sep 10 23:52:53.125040 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.125019    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d" event={"ID":"2875290f-2019-48de-a3f9-5fe2a880ee41","Type":"ContainerDied","Data":"9a4036a6e873ab35cbc0d207e41bfea57302db8e3b0e4aeee78e59df0e467fcd"}
Sep 10 23:52:53.125142 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.125047    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"
Sep 10 23:52:53.141421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.141403    2575 scope.go:117] "RemoveContainer" containerID="8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"
Sep 10 23:52:53.141734 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:52:53.141715    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa\": container with ID starting with 8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa not found: ID does not exist" containerID="8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"
Sep 10 23:52:53.141792 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.141743    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa"} err="failed to get container status \"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa\": rpc error: code = NotFound desc = could not find container \"8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa\": container with ID starting with 8707564d8c3733ce76a5a8e059510e0bb7470a30028d4fc3d78ba19365306eaa not found: ID does not exist"
Sep 10 23:52:53.141792 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.141759    2575 scope.go:117] "RemoveContainer" containerID="dc174b2066f2a8d534d0291b7efd8fdeab3575be25269b98864fddecd7cec597"
Sep 10 23:52:53.155632 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.155612    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:53.159076 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.159054    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-15a3a094-75d48b785c-z9svv"]
Sep 10 23:52:53.177063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.177041    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:53.180979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:53.180955    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-15a3a094-74bd676ffb-vlx5d"]
Sep 10 23:52:54.228692 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:54.228660    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2875290f-2019-48de-a3f9-5fe2a880ee41" path="/var/lib/kubelet/pods/2875290f-2019-48de-a3f9-5fe2a880ee41/volumes"
Sep 10 23:52:54.229206 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:52:54.229184    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" path="/var/lib/kubelet/pods/9a09bb65-dfdc-4a12-9c62-f6807e73996a/volumes"
Sep 10 23:53:46.062341 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.062306    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:53:46.062842 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.062586    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" podUID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" containerName="maas-api" containerID="cri-o://bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf" gracePeriod=30
Sep 10 23:53:46.121509 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.121466    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:53:46.121825 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.121768    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" podUID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" containerName="payload-pre-processing" containerID="cri-o://47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf" gracePeriod=30
Sep 10 23:53:46.364505 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.364463    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:53:46.396342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.396316    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:53:46.400910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.400875    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" containerID="47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf" exitCode=0
Sep 10 23:53:46.401035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.400914    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"
Sep 10 23:53:46.401035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.400949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" event={"ID":"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e","Type":"ContainerDied","Data":"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"}
Sep 10 23:53:46.401035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.400997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn" event={"ID":"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e","Type":"ContainerDied","Data":"98092a9403a215610df58ba20297aacc535eefaaf9a0c77a9245c65c3e3d884d"}
Sep 10 23:53:46.401035 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.401026    2575 scope.go:117] "RemoveContainer" containerID="47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"
Sep 10 23:53:46.402385 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.402337    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" containerID="bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf" exitCode=0
Sep 10 23:53:46.402385 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.402365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" event={"ID":"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc","Type":"ContainerDied","Data":"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"}
Sep 10 23:53:46.402385 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.402378    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"
Sep 10 23:53:46.402617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.402415    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5" event={"ID":"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc","Type":"ContainerDied","Data":"01d2c6a6ddbbb80728b8824b4ff4e7c2b38e51447abf4546c130f48fc6f8898a"}
Sep 10 23:53:46.410508 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.410475    2575 scope.go:117] "RemoveContainer" containerID="47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"
Sep 10 23:53:46.410891 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:53:46.410866    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf\": container with ID starting with 47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf not found: ID does not exist" containerID="47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"
Sep 10 23:53:46.410979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.410899    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf"} err="failed to get container status \"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf\": rpc error: code = NotFound desc = could not find container \"47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf\": container with ID starting with 47be687dde7f8f493bc33f889636fa015f4f6c0cccf52b4b23c56fae9efbdfdf not found: ID does not exist"
Sep 10 23:53:46.410979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.410919    2575 scope.go:117] "RemoveContainer" containerID="bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"
Sep 10 23:53:46.420859 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.420834    2575 scope.go:117] "RemoveContainer" containerID="bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"
Sep 10 23:53:46.421279 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:53:46.421219    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf\": container with ID starting with bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf not found: ID does not exist" containerID="bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"
Sep 10 23:53:46.421279 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.421253    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf"} err="failed to get container status \"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf\": rpc error: code = NotFound desc = could not find container \"bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf\": container with ID starting with bcb50727edf69b88be121e04e9a453612a3086fb266ae8f7ceff2f313fdfcfcf not found: ID does not exist"
Sep 10 23:53:46.478651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478616    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume\") pod \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") "
Sep 10 23:53:46.478651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478648    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs\") pod \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") "
Sep 10 23:53:46.478843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478666    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") pod \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") "
Sep 10 23:53:46.478843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478800    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbtkw\" (UniqueName: \"kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw\") pod \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\" (UID: \"d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e\") "
Sep 10 23:53:46.478843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478832    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca\") pod \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") "
Sep 10 23:53:46.479000 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.478889    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wccqf\" (UniqueName: \"kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf\") pod \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\" (UID: \"bbd5bb05-a0e2-49b5-999f-a44a2ea532cc\") "
Sep 10 23:53:46.479175 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.479020    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" (UID: "d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:53:46.479301 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.479185    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" (UID: "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:53:46.479393 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.479376    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.479452 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.479399    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-plugins-config-volume\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.480936 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.480916    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" (UID: "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:53:46.481015 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.480981    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf" (OuterVolumeSpecName: "kube-api-access-wccqf") pod "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" (UID: "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc"). InnerVolumeSpecName "kube-api-access-wccqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:53:46.481344 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.481327    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" (UID: "bbd5bb05-a0e2-49b5-999f-a44a2ea532cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:53:46.481406 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.481352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw" (OuterVolumeSpecName: "kube-api-access-gbtkw") pod "d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" (UID: "d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e"). InnerVolumeSpecName "kube-api-access-gbtkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:53:46.580386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.580305    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.580386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.580330    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.580386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.580338    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbtkw\" (UniqueName: \"kubernetes.io/projected/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e-kube-api-access-gbtkw\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.580386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.580347    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wccqf\" (UniqueName: \"kubernetes.io/projected/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc-kube-api-access-wccqf\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:53:46.723966 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.723938    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:53:46.730055 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.730030    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-677d3c-7b55b488f8-xs9sn"]
Sep 10 23:53:46.742097 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.742066    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:53:46.745079 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:46.745057    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-677d3c-db64867-mjrg5"]
Sep 10 23:53:48.228692 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:48.228666    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" path="/var/lib/kubelet/pods/bbd5bb05-a0e2-49b5-999f-a44a2ea532cc/volumes"
Sep 10 23:53:48.229057 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:53:48.229007    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" path="/var/lib/kubelet/pods/d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e/volumes"
Sep 10 23:54:20.031411 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031371    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031812    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" containerName="payload-processing"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031823    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" containerName="payload-processing"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031843    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" containerName="payload-pre-processing"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031850    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" containerName="payload-pre-processing"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031863    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2875290f-2019-48de-a3f9-5fe2a880ee41" containerName="maas-api"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031868    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2875290f-2019-48de-a3f9-5fe2a880ee41" containerName="maas-api"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031884    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" containerName="maas-api"
Sep 10 23:54:20.031911 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031889    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" containerName="maas-api"
Sep 10 23:54:20.032192 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031961    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7a8ae6f-0fdb-4647-ae46-5b959ba3db9e" containerName="payload-pre-processing"
Sep 10 23:54:20.032192 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031970    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2875290f-2019-48de-a3f9-5fe2a880ee41" containerName="maas-api"
Sep 10 23:54:20.032192 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031976    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a09bb65-dfdc-4a12-9c62-f6807e73996a" containerName="payload-processing"
Sep 10 23:54:20.032192 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.031982    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd5bb05-a0e2-49b5-999f-a44a2ea532cc" containerName="maas-api"
Sep 10 23:54:20.035114 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.035092    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.039074 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.039041    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-df1107\"/\"noten-model-df1107-kserve-self-signed-certs\""
Sep 10 23:54:20.039178 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.039089    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-df1107\"/\"default-dockercfg-92h2k\""
Sep 10 23:54:20.039178 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.039041    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-df1107\"/\"openshift-service-ca.crt\""
Sep 10 23:54:20.039178 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.039152    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-df1107\"/\"kube-root-ca.crt\""
Sep 10 23:54:20.044719 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.044696    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:20.088783 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088751    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.088925 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.088925 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-762w4\" (UniqueName: \"kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.089020 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.089020 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088963    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.089020 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.088978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190154 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190317 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-762w4\" (UniqueName: \"kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190317 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190241    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190317 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190317 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190575 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190767 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190744    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.190856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.190804    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.192882 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.192858    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.192882 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.192877    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.197106 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.197085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-762w4\" (UniqueName: \"kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4\") pod \"noten-model-df1107-kserve-747d6dbdb7-fl9h2\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") " pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.346405 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.346330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:20.618075 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:20.618050    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:20.619603 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:54:20.619572    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4dd9917_b35f_4a10_9de4_01729003ef7d.slice/crio-acc3225502dd7486c1868d334f019b38724703c87379e7ad285c96bd58270d56 WatchSource:0}: Error finding container acc3225502dd7486c1868d334f019b38724703c87379e7ad285c96bd58270d56: Status 404 returned error can't find the container with id acc3225502dd7486c1868d334f019b38724703c87379e7ad285c96bd58270d56
Sep 10 23:54:21.581342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:21.581306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" event={"ID":"b4dd9917-b35f-4a10-9de4-01729003ef7d","Type":"ContainerStarted","Data":"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"}
Sep 10 23:54:21.581342 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:21.581344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" event={"ID":"b4dd9917-b35f-4a10-9de4-01729003ef7d","Type":"ContainerStarted","Data":"acc3225502dd7486c1868d334f019b38724703c87379e7ad285c96bd58270d56"}
Sep 10 23:54:21.597819 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:21.597774    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" podStartSLOduration=1.5977600669999998 podStartE2EDuration="1.597760067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:54:21.596577282 +0000 UTC m=+1051.862574172" watchObservedRunningTime="2026-09-10 23:54:21.597760067 +0000 UTC m=+1051.863756937"
Sep 10 23:54:22.713779 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:22.713744    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:22.714205 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:22.713830    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:23.213808858 +0000 UTC m=+1053.479805714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:22.718188 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:22.718163    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:23.218633 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:23.218599    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:23.218809 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:23.218671    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:24.218654687 +0000 UTC m=+1054.484651535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:24.229879 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:24.229854    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:24.230255 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:24.229916    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:26.229901999 +0000 UTC m=+1056.495898847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:26.249289 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:26.249259    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:26.249700 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:26.249330    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:30.249316815 +0000 UTC m=+1060.515313664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:30.285996 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:30.285969    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:30.286395 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:30.286035    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:38.286022897 +0000 UTC m=+1068.552019749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:30.347312 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:30.347286    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:33.618072 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.618037    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:54:33.622995 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.622971    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.625901 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.625875    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ff0028b6-serving-cert\""
Sep 10 23:54:33.632037 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.632009    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:54:33.717177 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.717142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlk62\" (UniqueName: \"kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.717177 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.717177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.717395 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.717195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.717395 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.717274    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.818898 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.818862    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlk62\" (UniqueName: \"kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.818898 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.818907    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.819159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.818934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.819159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.819006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.819705 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.819681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.821529 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.821507    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.821624 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.821540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.831611 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.831590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlk62\" (UniqueName: \"kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62\") pod \"maas-api-e2e-mt-ff0028b6-8465768d94-54b7n\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:33.937783 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:33.937719    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:34.212137 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:34.212110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:54:34.214005 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:54:34.213978    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902b2def_24ff_4f13_8b61_7d617db66ed3.slice/crio-13ace95d70b2c99ff3f871e79ffda1d04e9f64c629628760d4a55fea6e6d0df7 WatchSource:0}: Error finding container 13ace95d70b2c99ff3f871e79ffda1d04e9f64c629628760d4a55fea6e6d0df7: Status 404 returned error can't find the container with id 13ace95d70b2c99ff3f871e79ffda1d04e9f64c629628760d4a55fea6e6d0df7
Sep 10 23:54:34.215412 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:34.215394    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:54:34.647910 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:34.647880    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" event={"ID":"902b2def-24ff-4f13-8b61-7d617db66ed3","Type":"ContainerStarted","Data":"13ace95d70b2c99ff3f871e79ffda1d04e9f64c629628760d4a55fea6e6d0df7"}
Sep 10 23:54:35.654300 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:35.654258    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" event={"ID":"902b2def-24ff-4f13-8b61-7d617db66ed3","Type":"ContainerStarted","Data":"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"}
Sep 10 23:54:35.654727 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:35.654344    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:54:35.672258 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:35.672208    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" podStartSLOduration=2.275135458 podStartE2EDuration="2.672190966s" totalImagesPullingTime="397.055508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:54:35.671035121 +0000 UTC m=+1065.937032003" watchObservedRunningTime="2026-09-10 23:54:35.672190966 +0000 UTC m=+1065.938187836"
Sep 10 23:54:38.363628 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:38.363585    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-self-signed-certs: secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:38.363993 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:38.363667    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs podName:b4dd9917-b35f-4a10-9de4-01729003ef7d nodeName:}" failed. No retries permitted until 2026-09-10 23:54:54.363651085 +0000 UTC m=+1084.629647933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs") pod "noten-model-df1107-kserve-747d6dbdb7-fl9h2" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d") : secret "noten-model-df1107-kserve-self-signed-certs" not found
Sep 10 23:54:38.611167 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.611111    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" podUID="b4dd9917-b35f-4a10-9de4-01729003ef7d" containerName="main" containerID="cri-o://1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820" gracePeriod=45
Sep 10 23:54:38.861872 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.861847    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:38.967628 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967549    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.967802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967647    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-762w4\" (UniqueName: \"kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.967802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967665    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.967802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967709    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.967802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967732    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.967802 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967802    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm\") pod \"b4dd9917-b35f-4a10-9de4-01729003ef7d\" (UID: \"b4dd9917-b35f-4a10-9de4-01729003ef7d\") "
Sep 10 23:54:38.968063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967841    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:38.968063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967978    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home" (OuterVolumeSpecName: "home") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:38.968063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.967991    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:38.968218 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.968193    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-tmp-dir\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:38.968269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.968220    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-home\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:38.968269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.968233    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-model-cache\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:38.970136 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.970108    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm" (OuterVolumeSpecName: "dshm") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:38.970256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.970155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:54:38.970256 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:38.970189    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4" (OuterVolumeSpecName: "kube-api-access-762w4") pod "b4dd9917-b35f-4a10-9de4-01729003ef7d" (UID: "b4dd9917-b35f-4a10-9de4-01729003ef7d"). InnerVolumeSpecName "kube-api-access-762w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:54:39.069693 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.069667    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4dd9917-b35f-4a10-9de4-01729003ef7d-dshm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:39.069693 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.069689    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-762w4\" (UniqueName: \"kubernetes.io/projected/b4dd9917-b35f-4a10-9de4-01729003ef7d-kube-api-access-762w4\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:39.069860 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.069699    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4dd9917-b35f-4a10-9de4-01729003ef7d-tls-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:54:39.675922 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.675893    2575 generic.go:356] "Generic (PLEG): container finished" podID="b4dd9917-b35f-4a10-9de4-01729003ef7d" containerID="1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820" exitCode=0
Sep 10 23:54:39.676361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.675941    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"
Sep 10 23:54:39.676361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.675975    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" event={"ID":"b4dd9917-b35f-4a10-9de4-01729003ef7d","Type":"ContainerDied","Data":"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"}
Sep 10 23:54:39.676361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.676032    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2" event={"ID":"b4dd9917-b35f-4a10-9de4-01729003ef7d","Type":"ContainerDied","Data":"acc3225502dd7486c1868d334f019b38724703c87379e7ad285c96bd58270d56"}
Sep 10 23:54:39.676361 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.676053    2575 scope.go:117] "RemoveContainer" containerID="1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"
Sep 10 23:54:39.685598 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.685583    2575 scope.go:117] "RemoveContainer" containerID="1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"
Sep 10 23:54:39.685849 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:54:39.685833    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820\": container with ID starting with 1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820 not found: ID does not exist" containerID="1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"
Sep 10 23:54:39.685922 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.685856    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820"} err="failed to get container status \"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820\": rpc error: code = NotFound desc = could not find container \"1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820\": container with ID starting with 1485c016ddd760c47a1cdd30084936aae4ab97cd2b071db949471b0bc316a820 not found: ID does not exist"
Sep 10 23:54:39.698799 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.698776    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:39.700958 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:39.700938    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-df1107/noten-model-df1107-kserve-747d6dbdb7-fl9h2"]
Sep 10 23:54:40.228600 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:40.228558    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4dd9917-b35f-4a10-9de4-01729003ef7d" path="/var/lib/kubelet/pods/b4dd9917-b35f-4a10-9de4-01729003ef7d/volumes"
Sep 10 23:54:41.664180 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:54:41.664097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:55:11.327183 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.327144    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:55:11.328119 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.328066    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" podUID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" containerName="maas-api" containerID="cri-o://79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85" gracePeriod=30
Sep 10 23:55:11.592099 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.592028    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:55:11.690748 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.690720    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5hq2\" (UniqueName: \"kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2\") pod \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") "
Sep 10 23:55:11.690917 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.690780    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca\") pod \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") "
Sep 10 23:55:11.690917 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.690816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls\") pod \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") "
Sep 10 23:55:11.690917 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.690869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs\") pod \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\" (UID: \"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3\") "
Sep 10 23:55:11.691183 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.691148    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" (UID: "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:55:11.693516 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.693315    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2" (OuterVolumeSpecName: "kube-api-access-b5hq2") pod "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" (UID: "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3"). InnerVolumeSpecName "kube-api-access-b5hq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:55:11.693516 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.693422    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" (UID: "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:11.693698 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.693486    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" (UID: "db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:11.792118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.792088    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:11.792118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.792114    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:11.792118 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.792123    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:11.792316 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.792131    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5hq2\" (UniqueName: \"kubernetes.io/projected/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3-kube-api-access-b5hq2\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:11.845063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.844985    2575 generic.go:356] "Generic (PLEG): container finished" podID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" containerID="79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85" exitCode=0
Sep 10 23:55:11.845063 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.845025    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"
Sep 10 23:55:11.845293 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.845067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" event={"ID":"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3","Type":"ContainerDied","Data":"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"}
Sep 10 23:55:11.845293 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.845128    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f" event={"ID":"db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3","Type":"ContainerDied","Data":"a001d6a15666cfea9d2a19c85745b9fe39a55c8b69b85dcd4759fe2b87841330"}
Sep 10 23:55:11.845293 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.845160    2575 scope.go:117] "RemoveContainer" containerID="79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"
Sep 10 23:55:11.855253 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.855235    2575 scope.go:117] "RemoveContainer" containerID="79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"
Sep 10 23:55:11.855607 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:55:11.855567    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85\": container with ID starting with 79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85 not found: ID does not exist" containerID="79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"
Sep 10 23:55:11.855682 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.855612    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85"} err="failed to get container status \"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85\": rpc error: code = NotFound desc = could not find container \"79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85\": container with ID starting with 79ae967342ff7eadf7145df6f64bcfea42e82b72a84332cfdbef47d83f0daa85 not found: ID does not exist"
Sep 10 23:55:11.870729 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.870701    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:55:11.880952 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:11.880928    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce1dd5-b78d8cc58-npq7f"]
Sep 10 23:55:12.227975 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:12.227886    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" path="/var/lib/kubelet/pods/db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3/volumes"
Sep 10 23:55:15.844751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:15.844710    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:55:15.845121 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:15.844975    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" podUID="902b2def-24ff-4f13-8b61-7d617db66ed3" containerName="maas-api" containerID="cri-o://37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118" gracePeriod=30
Sep 10 23:55:16.092522 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.092471    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:55:16.233811 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.233742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca\") pod \"902b2def-24ff-4f13-8b61-7d617db66ed3\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") "
Sep 10 23:55:16.233945 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.233845    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs\") pod \"902b2def-24ff-4f13-8b61-7d617db66ed3\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") "
Sep 10 23:55:16.233945 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.233896    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls\") pod \"902b2def-24ff-4f13-8b61-7d617db66ed3\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") "
Sep 10 23:55:16.233945 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.233938    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlk62\" (UniqueName: \"kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62\") pod \"902b2def-24ff-4f13-8b61-7d617db66ed3\" (UID: \"902b2def-24ff-4f13-8b61-7d617db66ed3\") "
Sep 10 23:55:16.234146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.234123    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "902b2def-24ff-4f13-8b61-7d617db66ed3" (UID: "902b2def-24ff-4f13-8b61-7d617db66ed3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:55:16.234248 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.234234    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902b2def-24ff-4f13-8b61-7d617db66ed3-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:16.236062 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.236023    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "902b2def-24ff-4f13-8b61-7d617db66ed3" (UID: "902b2def-24ff-4f13-8b61-7d617db66ed3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:16.236172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.236072    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62" (OuterVolumeSpecName: "kube-api-access-wlk62") pod "902b2def-24ff-4f13-8b61-7d617db66ed3" (UID: "902b2def-24ff-4f13-8b61-7d617db66ed3"). InnerVolumeSpecName "kube-api-access-wlk62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:55:16.236172 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.236115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "902b2def-24ff-4f13-8b61-7d617db66ed3" (UID: "902b2def-24ff-4f13-8b61-7d617db66ed3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:16.335031 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.335003    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:16.335031 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.335026    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902b2def-24ff-4f13-8b61-7d617db66ed3-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:16.335031 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.335034    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlk62\" (UniqueName: \"kubernetes.io/projected/902b2def-24ff-4f13-8b61-7d617db66ed3-kube-api-access-wlk62\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:16.875143 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.875105    2575 generic.go:356] "Generic (PLEG): container finished" podID="902b2def-24ff-4f13-8b61-7d617db66ed3" containerID="37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118" exitCode=0
Sep 10 23:55:16.875635 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.875164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" event={"ID":"902b2def-24ff-4f13-8b61-7d617db66ed3","Type":"ContainerDied","Data":"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"}
Sep 10 23:55:16.875635 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.875176    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"
Sep 10 23:55:16.875635 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.875250    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n" event={"ID":"902b2def-24ff-4f13-8b61-7d617db66ed3","Type":"ContainerDied","Data":"13ace95d70b2c99ff3f871e79ffda1d04e9f64c629628760d4a55fea6e6d0df7"}
Sep 10 23:55:16.875635 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.875284    2575 scope.go:117] "RemoveContainer" containerID="37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"
Sep 10 23:55:16.885029 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.885013    2575 scope.go:117] "RemoveContainer" containerID="37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"
Sep 10 23:55:16.885299 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:55:16.885279    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118\": container with ID starting with 37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118 not found: ID does not exist" containerID="37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"
Sep 10 23:55:16.885359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.885310    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118"} err="failed to get container status \"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118\": rpc error: code = NotFound desc = could not find container \"37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118\": container with ID starting with 37d141fef009d666ea2a52368e8cc6fdfaa033f5e5b76b9289719674a511d118 not found: ID does not exist"
Sep 10 23:55:16.895863 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.895841    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:55:16.899099 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:16.899079    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ff0028b6-8465768d94-54b7n"]
Sep 10 23:55:18.228456 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:18.228420    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="902b2def-24ff-4f13-8b61-7d617db66ed3" path="/var/lib/kubelet/pods/902b2def-24ff-4f13-8b61-7d617db66ed3/volumes"
Sep 10 23:55:29.000861 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.000831    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:55:29.001198 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.001063    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" podUID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" containerName="maas-api" containerID="cri-o://abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468" gracePeriod=30
Sep 10 23:55:29.254207 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.254150    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:55:29.258629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.258610    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") pod \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") "
Sep 10 23:55:29.258743 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.258727    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzc2f\" (UniqueName: \"kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f\") pod \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") "
Sep 10 23:55:29.258785 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.258765    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs\") pod \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") "
Sep 10 23:55:29.258843 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.258828    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca\") pod \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\" (UID: \"2bce7cfe-c6f6-4291-a048-9a462a3729cb\") "
Sep 10 23:55:29.259203 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.259178    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bce7cfe-c6f6-4291-a048-9a462a3729cb" (UID: "2bce7cfe-c6f6-4291-a048-9a462a3729cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:55:29.260029 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.259716    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bce7cfe-c6f6-4291-a048-9a462a3729cb-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:29.263616 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.263588    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bce7cfe-c6f6-4291-a048-9a462a3729cb" (UID: "2bce7cfe-c6f6-4291-a048-9a462a3729cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:29.263726 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.263612    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2bce7cfe-c6f6-4291-a048-9a462a3729cb" (UID: "2bce7cfe-c6f6-4291-a048-9a462a3729cb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:55:29.265675 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.265652    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f" (OuterVolumeSpecName: "kube-api-access-hzc2f") pod "2bce7cfe-c6f6-4291-a048-9a462a3729cb" (UID: "2bce7cfe-c6f6-4291-a048-9a462a3729cb"). InnerVolumeSpecName "kube-api-access-hzc2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:55:29.360485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.360449    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzc2f\" (UniqueName: \"kubernetes.io/projected/2bce7cfe-c6f6-4291-a048-9a462a3729cb-kube-api-access-hzc2f\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:29.360485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.360478    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:29.360485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.360511    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bce7cfe-c6f6-4291-a048-9a462a3729cb-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:55:29.951787 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.951750    2575 generic.go:356] "Generic (PLEG): container finished" podID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" containerID="abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468" exitCode=0
Sep 10 23:55:29.951997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.951805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" event={"ID":"2bce7cfe-c6f6-4291-a048-9a462a3729cb","Type":"ContainerDied","Data":"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"}
Sep 10 23:55:29.951997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.951812    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"
Sep 10 23:55:29.951997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.951869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s" event={"ID":"2bce7cfe-c6f6-4291-a048-9a462a3729cb","Type":"ContainerDied","Data":"d6c7d18c784bdae3c55bb2db97b15cd27d9651f2b5e7f7e657e06b65b16c6279"}
Sep 10 23:55:29.951997 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.951896    2575 scope.go:117] "RemoveContainer" containerID="abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"
Sep 10 23:55:29.961255 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.961236    2575 scope.go:117] "RemoveContainer" containerID="abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"
Sep 10 23:55:29.961570 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:55:29.961552    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468\": container with ID starting with abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468 not found: ID does not exist" containerID="abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"
Sep 10 23:55:29.961639 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.961582    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468"} err="failed to get container status \"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468\": rpc error: code = NotFound desc = could not find container \"abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468\": container with ID starting with abd0d7ed5db08a08860b0d87f33ae4e3b6408e03fdac779657cfda7fd501e468 not found: ID does not exist"
Sep 10 23:55:29.972210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.972188    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:55:29.976420 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:29.976396    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-13374d-5c6677db67-khn9s"]
Sep 10 23:55:30.227557 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:30.227468    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" path="/var/lib/kubelet/pods/2bce7cfe-c6f6-4291-a048-9a462a3729cb/volumes"
Sep 10 23:55:40.313006 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.312977    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 10 23:55:40.313429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313407    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" containerName="maas-api"
Sep 10 23:55:40.313429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313425    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" containerName="maas-api"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313439    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" containerName="maas-api"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313447    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" containerName="maas-api"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313464    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902b2def-24ff-4f13-8b61-7d617db66ed3" containerName="maas-api"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313475    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="902b2def-24ff-4f13-8b61-7d617db66ed3" containerName="maas-api"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313520    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4dd9917-b35f-4a10-9de4-01729003ef7d" containerName="main"
Sep 10 23:55:40.313596 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313529    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4dd9917-b35f-4a10-9de4-01729003ef7d" containerName="main"
Sep 10 23:55:40.313944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313613    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bce7cfe-c6f6-4291-a048-9a462a3729cb" containerName="maas-api"
Sep 10 23:55:40.313944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313626    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="902b2def-24ff-4f13-8b61-7d617db66ed3" containerName="maas-api"
Sep 10 23:55:40.313944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313635    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="db99fb18-3bc6-40d1-b4ab-99c3f58f3ef3" containerName="maas-api"
Sep 10 23:55:40.313944 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.313643    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4dd9917-b35f-4a10-9de4-01729003ef7d" containerName="main"
Sep 10 23:55:40.318373 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.318355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.320772 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.320751    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h7jzs\""
Sep 10 23:55:40.320946 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.320922    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-9eee5d-serving-cert\""
Sep 10 23:55:40.329203 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.329177    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 10 23:55:40.359759 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.359730    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.359898 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.359775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ct87\" (UniqueName: \"kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.359898 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.359839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.359991 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.359899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.461022 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.460993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.461152 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.461060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.461152 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.461139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.461223 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.461174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ct87\" (UniqueName: \"kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.461846 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.461820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.463770 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.463742    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.463979 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.463956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.472254 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.472236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ct87\" (UniqueName: \"kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87\") pod \"maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.631597 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.631478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:40.917108 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:40.917086    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 10 23:55:40.919360 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:55:40.919334    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08520636_f725_4d5f_8d71_e8e8578395dd.slice/crio-df60db790000abb531475f1e1dbb3b9cb2846dc44f69fef20f9d1ade7d2441da WatchSource:0}: Error finding container df60db790000abb531475f1e1dbb3b9cb2846dc44f69fef20f9d1ade7d2441da: Status 404 returned error can't find the container with id df60db790000abb531475f1e1dbb3b9cb2846dc44f69fef20f9d1ade7d2441da
Sep 10 23:55:41.013657 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.013626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" event={"ID":"08520636-f725-4d5f-8d71-e8e8578395dd","Type":"ContainerStarted","Data":"df60db790000abb531475f1e1dbb3b9cb2846dc44f69fef20f9d1ade7d2441da"}
Sep 10 23:55:41.316447 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.316404    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 10 23:55:41.320656 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.320632    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.323433 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.323412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-37692b-serving-cert\""
Sep 10 23:55:41.350628 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.350604    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 10 23:55:41.369508 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.369473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.369660 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.369609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbn6\" (UniqueName: \"kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.369732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.369685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.369928 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.369764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.471245 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.471207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.471409 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.471302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.471409 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.471365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.471546 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.471460    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbn6\" (UniqueName: \"kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.471967 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.471941    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.473810 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.473786    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.473892 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.473865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.491007 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.490983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbn6\" (UniqueName: \"kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6\") pod \"maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.639435 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.639350    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:41.921740 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:41.921713    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 10 23:55:41.924272 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:55:41.924244    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2d1537_3398_4152_b3ef_79e5bdb8aa9d.slice/crio-b08666dd063d1b3e0494de3ed9a951e1b7c800f6756778df53b19e01d81e38b2 WatchSource:0}: Error finding container b08666dd063d1b3e0494de3ed9a951e1b7c800f6756778df53b19e01d81e38b2: Status 404 returned error can't find the container with id b08666dd063d1b3e0494de3ed9a951e1b7c800f6756778df53b19e01d81e38b2
Sep 10 23:55:42.020810 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:42.020774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" event={"ID":"08520636-f725-4d5f-8d71-e8e8578395dd","Type":"ContainerStarted","Data":"e675fbca5b4e5e07a32aed63dfbf09c2e18da655817a2214e0adcebfd6e0781b"}
Sep 10 23:55:42.021033 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:42.020838    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:42.022086 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:42.022059    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" event={"ID":"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d","Type":"ContainerStarted","Data":"b08666dd063d1b3e0494de3ed9a951e1b7c800f6756778df53b19e01d81e38b2"}
Sep 10 23:55:42.040392 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:42.040347    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" podStartSLOduration=1.6305741569999999 podStartE2EDuration="2.040330056s" totalImagesPullingTime="409.755899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:42.03804147 +0000 UTC m=+1132.304038353" watchObservedRunningTime="2026-09-10 23:55:42.040330056 +0000 UTC m=+1132.306326927"
Sep 10 23:55:43.030194 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:43.030147    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" event={"ID":"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d","Type":"ContainerStarted","Data":"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"}
Sep 10 23:55:43.030754 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:43.030366    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:55:43.048931 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:43.048878    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" podStartSLOduration=1.743950575 podStartE2EDuration="2.048862308s" totalImagesPullingTime="304.911733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:43.048204493 +0000 UTC m=+1133.314201388" watchObservedRunningTime="2026-09-10 23:55:43.048862308 +0000 UTC m=+1133.314859178"
Sep 10 23:55:48.035806 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:48.035773    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 10 23:55:49.040867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:55:49.040835    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 10 23:56:09.213051 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.213021    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 10 23:56:09.218081 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.218062    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.220561 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.220541    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-dd5958-serving-cert\""
Sep 10 23:56:09.226416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.226353    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 10 23:56:09.339026 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.338986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.339211 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.339185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5p4\" (UniqueName: \"kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.339269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.339234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.339318 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.339304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.440730 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.440688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5p4\" (UniqueName: \"kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.440900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.440740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.440900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.440787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.440900 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.440827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.441507 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.441429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.443416 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.443393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.443540 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.443434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.449238 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.449219    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5p4\" (UniqueName: \"kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4\") pod \"maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.529337 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.529290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:09.816525 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:09.816483    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 10 23:56:09.819236 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:56:09.819208    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa69bef5_cf8d_49b2_9bc6_877a15826a4d.slice/crio-1539f4c61081e7fff2f648b47041ae43f7ffa032907216cc1dd3c656ea2fa91c WatchSource:0}: Error finding container 1539f4c61081e7fff2f648b47041ae43f7ffa032907216cc1dd3c656ea2fa91c: Status 404 returned error can't find the container with id 1539f4c61081e7fff2f648b47041ae43f7ffa032907216cc1dd3c656ea2fa91c
Sep 10 23:56:10.179097 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:10.179013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" event={"ID":"aa69bef5-cf8d-49b2-9bc6-877a15826a4d","Type":"ContainerStarted","Data":"1539f4c61081e7fff2f648b47041ae43f7ffa032907216cc1dd3c656ea2fa91c"}
Sep 10 23:56:11.011346 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.011315    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 10 23:56:11.015098 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.015079    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.017201 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.017180    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-e96d4c-serving-cert\""
Sep 10 23:56:11.024480 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.024456    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 10 23:56:11.054198 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.054168    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjpp2\" (UniqueName: \"kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.054354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.054223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.054354 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.054315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.054456 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.054371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.154948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.154918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjpp2\" (UniqueName: \"kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.154948 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.154959    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.155166 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.154989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.155166 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.155021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.155166 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:56:11.155127    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-serving-cert: secret "maas-api-e2e-shared-b-w3-e96d4c-serving-cert" not found
Sep 10 23:56:11.155286 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:56:11.155213    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls podName:da110c99-f0a9-463c-8351-c2ec9f59ad15 nodeName:}" failed. No retries permitted until 2026-09-10 23:56:11.655191058 +0000 UTC m=+1161.921187912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls") pod "maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" (UID: "da110c99-f0a9-463c-8351-c2ec9f59ad15") : secret "maas-api-e2e-shared-b-w3-e96d4c-serving-cert" not found
Sep 10 23:56:11.155629 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.155613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.157527 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.157511    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.163386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.163366    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjpp2\" (UniqueName: \"kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.185749 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.185720    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" event={"ID":"aa69bef5-cf8d-49b2-9bc6-877a15826a4d","Type":"ContainerStarted","Data":"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"}
Sep 10 23:56:11.185867 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.185809    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:11.206013 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.205962    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" podStartSLOduration=1.694483552 podStartE2EDuration="2.205948768s" totalImagesPullingTime="511.465216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:56:11.203998077 +0000 UTC m=+1161.469994975" watchObservedRunningTime="2026-09-10 23:56:11.205948768 +0000 UTC m=+1161.471945637"
Sep 10 23:56:11.659421 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.659347    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.661896 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.661867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:11.927429 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:11.927344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:12.200746 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:12.200724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 10 23:56:12.202887 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:56:12.202859    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda110c99_f0a9_463c_8351_c2ec9f59ad15.slice/crio-11ed70d6681c74e42e6fb935b20fd3ee0795edb3535fb62a579e2271a56364ee WatchSource:0}: Error finding container 11ed70d6681c74e42e6fb935b20fd3ee0795edb3535fb62a579e2271a56364ee: Status 404 returned error can't find the container with id 11ed70d6681c74e42e6fb935b20fd3ee0795edb3535fb62a579e2271a56364ee
Sep 10 23:56:13.196689 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:13.196658    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" event={"ID":"da110c99-f0a9-463c-8351-c2ec9f59ad15","Type":"ContainerStarted","Data":"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"}
Sep 10 23:56:13.196890 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:13.196714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" event={"ID":"da110c99-f0a9-463c-8351-c2ec9f59ad15","Type":"ContainerStarted","Data":"11ed70d6681c74e42e6fb935b20fd3ee0795edb3535fb62a579e2271a56364ee"}
Sep 10 23:56:13.196890 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:13.196812    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:13.220829 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:13.220778    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" podStartSLOduration=2.7620333329999998 podStartE2EDuration="3.220763523s" totalImagesPullingTime="458.73019ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:56:13.21917797 +0000 UTC m=+1163.485174864" watchObservedRunningTime="2026-09-10 23:56:13.220763523 +0000 UTC m=+1163.486760392"
Sep 10 23:56:17.195642 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:17.195613    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 10 23:56:19.208052 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:19.208023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 10 23:56:46.598351 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.598316    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:56:46.603651 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.603627    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.606310 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.606286    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6c869d5a-serving-cert\""
Sep 10 23:56:46.613140 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.613117    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:56:46.800437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.800405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.800437 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.800437    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzbv\" (UniqueName: \"kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.800704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.800481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.800704 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.800609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.901745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.901668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.901745 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.901738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.901939 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.901756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzbv\" (UniqueName: \"kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.901939 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:56:46.901826    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-serving-cert: secret "maas-api-e2e-mt-6c869d5a-serving-cert" not found
Sep 10 23:56:46.901939 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.901858    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.901939 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:56:46.901898    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls podName:d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca nodeName:}" failed. No retries permitted until 2026-09-10 23:56:47.401880119 +0000 UTC m=+1197.667876980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls") pod "maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" (UID: "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca") : secret "maas-api-e2e-mt-6c869d5a-serving-cert" not found
Sep 10 23:56:46.902422 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.902406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.904438 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.904415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:46.921159 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:46.921131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzbv\" (UniqueName: \"kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:47.408308 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:47.408270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:47.410784 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:47.410756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") pod \"maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:47.516960 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:47.516927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:47.822684 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:47.822658    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:56:47.824974 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:56:47.824948    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2119d1d_7dcd_4cb0_bdd7_5a64ba8a65ca.slice/crio-c1304470e4029fc2c088817068a5d5ee97acdbac687fcd530bfb1b0423e39cf9 WatchSource:0}: Error finding container c1304470e4029fc2c088817068a5d5ee97acdbac687fcd530bfb1b0423e39cf9: Status 404 returned error can't find the container with id c1304470e4029fc2c088817068a5d5ee97acdbac687fcd530bfb1b0423e39cf9
Sep 10 23:56:48.381705 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:48.381665    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" event={"ID":"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca","Type":"ContainerStarted","Data":"f1017bc3e0ea5dc3a7c8a988a60a40cae6dc3412461be3e4327af23f65f56928"}
Sep 10 23:56:48.381975 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:48.381739    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" event={"ID":"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca","Type":"ContainerStarted","Data":"c1304470e4029fc2c088817068a5d5ee97acdbac687fcd530bfb1b0423e39cf9"}
Sep 10 23:56:48.381975 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:48.381784    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:56:48.399661 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:48.399603    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" podStartSLOduration=2.006143774 podStartE2EDuration="2.399581446s" totalImagesPullingTime="393.437672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:56:48.397463409 +0000 UTC m=+1198.663460301" watchObservedRunningTime="2026-09-10 23:56:48.399581446 +0000 UTC m=+1198.665578318"
Sep 10 23:56:54.392548 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:56:54.392518    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:57:17.536617 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.536584    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 10 23:57:17.541665 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.541645    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.545345 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.545322    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"default-dockercfg-czf62\""
Sep 10 23:57:17.545485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.545350    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"kube-root-ca.crt\""
Sep 10 23:57:17.545485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.545359    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"openshift-service-ca.crt\""
Sep 10 23:57:17.545485 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.545329    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"auth-test-model-e96d4c-kserve-self-signed-certs\""
Sep 10 23:57:17.558698 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.558670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 10 23:57:17.693210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.693425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brhcp\" (UniqueName: \"kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.693425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.693425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.693425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.693425 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.693426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794171 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794171 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brhcp\" (UniqueName: \"kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794359 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794575 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794683 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.794732 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.794667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.796543 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.796519    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.796875 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.796856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.802720 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.802700    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brhcp\" (UniqueName: \"kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp\") pod \"auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:17.853888 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:17.853855    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:18.125833 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:18.125804    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 10 23:57:18.127366 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:57:18.127340    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc446fa6_433a_43f9_a158_05f9142fed10.slice/crio-7065e6bce63bf7c4cf0016b3d7c09c2b6a2996c53304dc8e7cf5891589f80b98 WatchSource:0}: Error finding container 7065e6bce63bf7c4cf0016b3d7c09c2b6a2996c53304dc8e7cf5891589f80b98: Status 404 returned error can't find the container with id 7065e6bce63bf7c4cf0016b3d7c09c2b6a2996c53304dc8e7cf5891589f80b98
Sep 10 23:57:18.546261 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:18.546224    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" event={"ID":"dc446fa6-433a-43f9-a158-05f9142fed10","Type":"ContainerStarted","Data":"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"}
Sep 10 23:57:18.546652 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:18.546271    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" event={"ID":"dc446fa6-433a-43f9-a158-05f9142fed10","Type":"ContainerStarted","Data":"7065e6bce63bf7c4cf0016b3d7c09c2b6a2996c53304dc8e7cf5891589f80b98"}
Sep 10 23:57:18.563207 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:18.563160    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" podStartSLOduration=1.5631450949999999 podStartE2EDuration="1.563145095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:57:18.562201286 +0000 UTC m=+1228.828198353" watchObservedRunningTime="2026-09-10 23:57:18.563145095 +0000 UTC m=+1228.829141964"
Sep 10 23:57:27.854951 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:27.854917    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:27.854951 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:27.854958    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:27.867252 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:27.867229    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:28.609856 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:28.609827    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 10 23:57:47.729517 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.729420    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:57:47.732931 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.732915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.735390 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.735363    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6c869d5a\"/\"kube-root-ca.crt\""
Sep 10 23:57:47.736327 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.736304    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6c869d5a\"/\"e2e-lifecycle-model-6c869d5a-kserve-self-signed-certs\""
Sep 10 23:57:47.736470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.736333    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6c869d5a\"/\"default-dockercfg-hb4wx\""
Sep 10 23:57:47.736470 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.736362    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6c869d5a\"/\"openshift-service-ca.crt\""
Sep 10 23:57:47.742375 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.742349    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:57:47.788353 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvz65\" (UniqueName: \"kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.788528 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.788606 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.788606 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.788712 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.788712 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.788674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890017 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.889981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvz65\" (UniqueName: \"kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890210 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890522 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890586 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.890586 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.890570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.892471 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.892452    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.892777 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.892758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:47.898571 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:47.898550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvz65\" (UniqueName: \"kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65\") pod \"e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") " pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:48.044403 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:48.044372    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:48.314809 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:48.314784    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:57:48.317324 ip-10-0-134-115 kubenswrapper[2575]: W0910 23:57:48.317293    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64477330_dbe7_4a7f_bbfd_28dbf2cc98d6.slice/crio-b257308bbf45563bf3cc387f22846a0ae654a5f5f9334226688fb49a2de3b55c WatchSource:0}: Error finding container b257308bbf45563bf3cc387f22846a0ae654a5f5f9334226688fb49a2de3b55c: Status 404 returned error can't find the container with id b257308bbf45563bf3cc387f22846a0ae654a5f5f9334226688fb49a2de3b55c
Sep 10 23:57:48.700724 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:48.700638    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" event={"ID":"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6","Type":"ContainerStarted","Data":"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"}
Sep 10 23:57:48.700724 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:48.700682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" event={"ID":"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6","Type":"ContainerStarted","Data":"b257308bbf45563bf3cc387f22846a0ae654a5f5f9334226688fb49a2de3b55c"}
Sep 10 23:57:48.721309 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:48.721254    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" podStartSLOduration=1.721240008 podStartE2EDuration="1.721240008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:57:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:57:48.718093723 +0000 UTC m=+1258.984090606" watchObservedRunningTime="2026-09-10 23:57:48.721240008 +0000 UTC m=+1258.987236877"
Sep 10 23:57:58.045092 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:58.045058    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:58.045468 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:58.045106    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:58.056999 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:58.056974    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:57:58.770077 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:57:58.770046    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:58:40.847001 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:40.846965    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:58:40.847595 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:40.847260    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" podUID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" containerName="maas-api" containerID="cri-o://f1017bc3e0ea5dc3a7c8a988a60a40cae6dc3412461be3e4327af23f65f56928" gracePeriod=30
Sep 10 23:58:40.981578 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:40.981542    2575 generic.go:356] "Generic (PLEG): container finished" podID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" containerID="f1017bc3e0ea5dc3a7c8a988a60a40cae6dc3412461be3e4327af23f65f56928" exitCode=0
Sep 10 23:58:40.981751 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:40.981623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" event={"ID":"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca","Type":"ContainerDied","Data":"f1017bc3e0ea5dc3a7c8a988a60a40cae6dc3412461be3e4327af23f65f56928"}
Sep 10 23:58:41.099814 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.099749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:58:41.205943 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.205913    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs\") pod \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") "
Sep 10 23:58:41.206146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.205988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") pod \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") "
Sep 10 23:58:41.206146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.206031    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca\") pod \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") "
Sep 10 23:58:41.206146 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.206099    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bzbv\" (UniqueName: \"kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv\") pod \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\" (UID: \"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca\") "
Sep 10 23:58:41.206450 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.206423    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" (UID: "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:58:41.208438 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.208409    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" (UID: "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:58:41.208640 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.208621    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" (UID: "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:58:41.208742 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.208716    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv" (OuterVolumeSpecName: "kube-api-access-8bzbv") pod "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" (UID: "d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca"). InnerVolumeSpecName "kube-api-access-8bzbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:58:41.307110 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.307081    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:58:41.307110 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.307105    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:58:41.307110 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.307115    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:58:41.307314 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.307125    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bzbv\" (UniqueName: \"kubernetes.io/projected/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca-kube-api-access-8bzbv\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:58:41.988667 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.988637    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"
Sep 10 23:58:41.989135 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.988633    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t" event={"ID":"d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca","Type":"ContainerDied","Data":"c1304470e4029fc2c088817068a5d5ee97acdbac687fcd530bfb1b0423e39cf9"}
Sep 10 23:58:41.989135 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:41.988776    2575 scope.go:117] "RemoveContainer" containerID="f1017bc3e0ea5dc3a7c8a988a60a40cae6dc3412461be3e4327af23f65f56928"
Sep 10 23:58:42.010378 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:42.010342    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:58:42.012386 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:42.012366    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c869d5a-67ff4d7f5d-b5v9t"]
Sep 10 23:58:42.228467 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:42.228432    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" path="/var/lib/kubelet/pods/d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca/volumes"
Sep 10 23:58:49.104021 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:58:49.103978    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:59:04.123971 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.123917    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" podUID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" containerName="main" containerID="cri-o://24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43" gracePeriod=45
Sep 10 23:59:04.370329 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.370305    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:59:04.429108 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429031    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429108 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429090    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429129    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429200    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvz65\" (UniqueName: \"kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429269 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429261    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429311    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home" (OuterVolumeSpecName: "home") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:59:04.429387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429327    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:59:04.429387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429343    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs\") pod \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\" (UID: \"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6\") "
Sep 10 23:59:04.429557 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429538    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:59:04.429725 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429709    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tmp-dir\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:04.429787 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429726    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-home\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:04.429787 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.429737    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-model-cache\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:04.431598 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.431573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65" (OuterVolumeSpecName: "kube-api-access-vvz65") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "kube-api-access-vvz65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:59:04.431705 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.431631    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:59:04.431749 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.431714    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm" (OuterVolumeSpecName: "dshm") pod "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" (UID: "64477330-dbe7-4a7f-bbfd-28dbf2cc98d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:59:04.530202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.530171    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-tls-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:04.530202 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.530198    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-dshm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:04.530347 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:04.530211    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvz65\" (UniqueName: \"kubernetes.io/projected/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6-kube-api-access-vvz65\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 10 23:59:05.112387 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.112355    2575 generic.go:356] "Generic (PLEG): container finished" podID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" containerID="24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43" exitCode=0
Sep 10 23:59:05.112589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.112399    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"
Sep 10 23:59:05.112589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.112394    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" event={"ID":"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6","Type":"ContainerDied","Data":"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"}
Sep 10 23:59:05.112589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.112537    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j" event={"ID":"64477330-dbe7-4a7f-bbfd-28dbf2cc98d6","Type":"ContainerDied","Data":"b257308bbf45563bf3cc387f22846a0ae654a5f5f9334226688fb49a2de3b55c"}
Sep 10 23:59:05.112589 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.112557    2575 scope.go:117] "RemoveContainer" containerID="24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"
Sep 10 23:59:05.128424 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.128222    2575 scope.go:117] "RemoveContainer" containerID="24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"
Sep 10 23:59:05.128688 ip-10-0-134-115 kubenswrapper[2575]: E0910 23:59:05.128538    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43\": container with ID starting with 24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43 not found: ID does not exist" containerID="24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"
Sep 10 23:59:05.128688 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.128561    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43"} err="failed to get container status \"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43\": rpc error: code = NotFound desc = could not find container \"24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43\": container with ID starting with 24385d3d115dfe212d471910dd651f09ecb86f337327560da7d473514c10af43 not found: ID does not exist"
Sep 10 23:59:05.136277 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.136256    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:59:05.139601 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:05.139580    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6c869d5a/e2e-lifecycle-model-6c869d5a-kserve-6f879495c9-q646j"]
Sep 10 23:59:06.228528 ip-10-0-134-115 kubenswrapper[2575]: I0910 23:59:06.228473    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" path="/var/lib/kubelet/pods/64477330-dbe7-4a7f-bbfd-28dbf2cc98d6/volumes"
Sep 11 00:00:00.259868 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.259828    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29818080-s6gs7"]
Sep 11 00:00:00.260305 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260261    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" containerName="main"
Sep 11 00:00:00.260305 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260272    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" containerName="main"
Sep 11 00:00:00.260305 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260286    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" containerName="maas-api"
Sep 11 00:00:00.260305 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260293    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" containerName="maas-api"
Sep 11 00:00:00.260431 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260353    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="64477330-dbe7-4a7f-bbfd-28dbf2cc98d6" containerName="main"
Sep 11 00:00:00.260431 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.260362    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2119d1d-7dcd-4cb0-bdd7-5a64ba8a65ca" containerName="maas-api"
Sep 11 00:00:00.264701 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.264686    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.267706 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.267682    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Sep 11 00:00:00.267838 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.267743    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-rk97b\""
Sep 11 00:00:00.277453 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.277427    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29818080-s6gs7"]
Sep 11 00:00:00.370637 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.370605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.370780 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.370760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v95b9\" (UniqueName: \"kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.471324 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.471292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v95b9\" (UniqueName: \"kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.471526 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.471376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.471978 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.471960    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.485104 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.485079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v95b9\" (UniqueName: \"kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9\") pod \"image-pruner-29818080-s6gs7\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") " pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.592668 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.592638    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:00.865142 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:00.865063    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29818080-s6gs7"]
Sep 11 00:00:00.868184 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:00:00.868160    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931e2487_bf61_45d1_b887_a0732a343188.slice/crio-0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c WatchSource:0}: Error finding container 0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c: Status 404 returned error can't find the container with id 0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c
Sep 11 00:00:01.395037 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:01.395005    2575 generic.go:356] "Generic (PLEG): container finished" podID="931e2487-bf61-45d1-b887-a0732a343188" containerID="166b26ecabc6d326e3529bef714d129c42e4b9367a77f230a43c3dd181a19e32" exitCode=0
Sep 11 00:00:01.395537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:01.395100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29818080-s6gs7" event={"ID":"931e2487-bf61-45d1-b887-a0732a343188","Type":"ContainerDied","Data":"166b26ecabc6d326e3529bef714d129c42e4b9367a77f230a43c3dd181a19e32"}
Sep 11 00:00:01.395537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:01.395168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29818080-s6gs7" event={"ID":"931e2487-bf61-45d1-b887-a0732a343188","Type":"ContainerStarted","Data":"0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c"}
Sep 11 00:00:02.540778 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.540753    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:02.695258 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.695176    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v95b9\" (UniqueName: \"kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9\") pod \"931e2487-bf61-45d1-b887-a0732a343188\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") "
Sep 11 00:00:02.695405 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.695283    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca\") pod \"931e2487-bf61-45d1-b887-a0732a343188\" (UID: \"931e2487-bf61-45d1-b887-a0732a343188\") "
Sep 11 00:00:02.695807 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.695774    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca" (OuterVolumeSpecName: "serviceca") pod "931e2487-bf61-45d1-b887-a0732a343188" (UID: "931e2487-bf61-45d1-b887-a0732a343188"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:00:02.697504 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.697463    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9" (OuterVolumeSpecName: "kube-api-access-v95b9") pod "931e2487-bf61-45d1-b887-a0732a343188" (UID: "931e2487-bf61-45d1-b887-a0732a343188"). InnerVolumeSpecName "kube-api-access-v95b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:00:02.796259 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.796227    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v95b9\" (UniqueName: \"kubernetes.io/projected/931e2487-bf61-45d1-b887-a0732a343188-kube-api-access-v95b9\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:02.796259 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:02.796259    2575 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/931e2487-bf61-45d1-b887-a0732a343188-serviceca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:03.407510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:03.407478    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29818080-s6gs7"
Sep 11 00:00:03.407698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:03.407517    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29818080-s6gs7" event={"ID":"931e2487-bf61-45d1-b887-a0732a343188","Type":"ContainerDied","Data":"0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c"}
Sep 11 00:00:03.407698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:03.407561    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0aad6d2d804cd693b6672b1bbfad15930b8a31973e660bead9ac802ba4c3c99c"
Sep 11 00:00:03.752015 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:03.751937    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 11 00:00:18.771043 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:18.770980    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" podUID="dc446fa6-433a-43f9-a158-05f9142fed10" containerName="main" containerID="cri-o://22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50" gracePeriod=45
Sep 11 00:00:19.027151 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.027096    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 11 00:00:19.167709 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167679    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.167860 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.167860 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167763    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.167860 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167787    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.167860 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167812    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brhcp\" (UniqueName: \"kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.168088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167884    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs\") pod \"dc446fa6-433a-43f9-a158-05f9142fed10\" (UID: \"dc446fa6-433a-43f9-a158-05f9142fed10\") "
Sep 11 00:00:19.168088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.167928    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:00:19.168088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.168041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:00:19.168088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.168056    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home" (OuterVolumeSpecName: "home") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:00:19.168264 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.168192    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-tmp-dir\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.168264 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.168206    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-home\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.168264 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.168214    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-model-cache\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.170214 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.170191    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm" (OuterVolumeSpecName: "dshm") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:00:19.170305 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.170223    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:00:19.170362 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.170327    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp" (OuterVolumeSpecName: "kube-api-access-brhcp") pod "dc446fa6-433a-43f9-a158-05f9142fed10" (UID: "dc446fa6-433a-43f9-a158-05f9142fed10"). InnerVolumeSpecName "kube-api-access-brhcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:00:19.269748 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.269713    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc446fa6-433a-43f9-a158-05f9142fed10-dshm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.269748 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.269745    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brhcp\" (UniqueName: \"kubernetes.io/projected/dc446fa6-433a-43f9-a158-05f9142fed10-kube-api-access-brhcp\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.269972 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.269759    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc446fa6-433a-43f9-a158-05f9142fed10-tls-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:00:19.492763 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.492726    2575 generic.go:356] "Generic (PLEG): container finished" podID="dc446fa6-433a-43f9-a158-05f9142fed10" containerID="22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50" exitCode=0
Sep 11 00:00:19.492939 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.492788    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"
Sep 11 00:00:19.492939 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.492816    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" event={"ID":"dc446fa6-433a-43f9-a158-05f9142fed10","Type":"ContainerDied","Data":"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"}
Sep 11 00:00:19.492939 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.492865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf" event={"ID":"dc446fa6-433a-43f9-a158-05f9142fed10","Type":"ContainerDied","Data":"7065e6bce63bf7c4cf0016b3d7c09c2b6a2996c53304dc8e7cf5891589f80b98"}
Sep 11 00:00:19.492939 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.492888    2575 scope.go:117] "RemoveContainer" containerID="22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"
Sep 11 00:00:19.504417 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.504401    2575 scope.go:117] "RemoveContainer" containerID="22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"
Sep 11 00:00:19.504704 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:00:19.504687    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50\": container with ID starting with 22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50 not found: ID does not exist" containerID="22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"
Sep 11 00:00:19.504763 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.504711    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50"} err="failed to get container status \"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50\": rpc error: code = NotFound desc = could not find container \"22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50\": container with ID starting with 22e0dea3c0639d1d2ed1477fe8bf1f96b9d1d241fbf3c34b4c6c42c4c0945c50 not found: ID does not exist"
Sep 11 00:00:19.515134 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.515111    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 11 00:00:19.517051 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:19.517033    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/auth-test-model-e96d4c-kserve-5c4f474fff-hxcvf"]
Sep 11 00:00:20.229898 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:20.229868    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc446fa6-433a-43f9-a158-05f9142fed10" path="/var/lib/kubelet/pods/dc446fa6-433a-43f9-a158-05f9142fed10/volumes"
Sep 11 00:00:30.931076 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931046    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:00:30.931510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931475    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931e2487-bf61-45d1-b887-a0732a343188" containerName="image-pruner"
Sep 11 00:00:30.931510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931504    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="931e2487-bf61-45d1-b887-a0732a343188" containerName="image-pruner"
Sep 11 00:00:30.931510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931513    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc446fa6-433a-43f9-a158-05f9142fed10" containerName="main"
Sep 11 00:00:30.931698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931518    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc446fa6-433a-43f9-a158-05f9142fed10" containerName="main"
Sep 11 00:00:30.931698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931584    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="931e2487-bf61-45d1-b887-a0732a343188" containerName="image-pruner"
Sep 11 00:00:30.931698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.931597    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc446fa6-433a-43f9-a158-05f9142fed10" containerName="main"
Sep 11 00:00:30.935973 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.935957    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:30.939325 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.939303    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"default-dockercfg-czf62\""
Sep 11 00:00:30.939439 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.939365    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"sub-test-model-e96d4c-kserve-self-signed-certs\""
Sep 11 00:00:30.939439 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.939307    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"kube-root-ca.crt\""
Sep 11 00:00:30.939439 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.939315    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e96d4c\"/\"openshift-service-ca.crt\""
Sep 11 00:00:30.950296 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:30.950272    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:00:31.081453 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081420    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.081631 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.081631 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.081631 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.081631 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnlq\" (UniqueName: \"kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.081770 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.081651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.103469 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.103437    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:00:31.107316 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.107297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.110034 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.110010    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-aef6df-serving-cert\""
Sep 11 00:00:31.122141 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.122110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:00:31.182807 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.182725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.182807 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.182784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.182996 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.182865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.182996 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.182892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.182996 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.182936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnlq\" (UniqueName: \"kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.183114 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.183001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.183114 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.183099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.183279 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.183261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.183764 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.183735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.185511 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.185456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.186297 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.186272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.197790 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.197768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnlq\" (UniqueName: \"kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq\") pod \"sub-test-model-e96d4c-kserve-7d974f6457-4tnsk\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") " pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.245956 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.245923    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:31.284095 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.283783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.284095 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.283879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.284095 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.283926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.284095 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.284000    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ndvp\" (UniqueName: \"kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.385087 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.385052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.385238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.385124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.385238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.385151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.385238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.385202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ndvp\" (UniqueName: \"kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.386006 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.385983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.387661 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.387641    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.387717 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.387665    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.392790 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.392767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ndvp\" (UniqueName: \"kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp\") pod \"maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.418362 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.418340    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:31.517692 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.517660    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:00:31.519112 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:00:31.519086    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5c28768_6b40_4b1c_bf4d_119a4b9e6950.slice/crio-6611fa34a9f1a8d1f7ee11a5dcd13758eaa5479d2511ce14270064bc8be1a53b WatchSource:0}: Error finding container 6611fa34a9f1a8d1f7ee11a5dcd13758eaa5479d2511ce14270064bc8be1a53b: Status 404 returned error can't find the container with id 6611fa34a9f1a8d1f7ee11a5dcd13758eaa5479d2511ce14270064bc8be1a53b
Sep 11 00:00:31.558422 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.558394    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" event={"ID":"d5c28768-6b40-4b1c-bf4d-119a4b9e6950","Type":"ContainerStarted","Data":"6611fa34a9f1a8d1f7ee11a5dcd13758eaa5479d2511ce14270064bc8be1a53b"}
Sep 11 00:00:31.698837 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.698816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:00:31.701043 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:00:31.701016    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08334e8_6218_4f8c_bb17_a21b1c881368.slice/crio-80d941ea8ed2b680aff060980b7bb2b5789775e18070999d9f1775c60dbcbbb8 WatchSource:0}: Error finding container 80d941ea8ed2b680aff060980b7bb2b5789775e18070999d9f1775c60dbcbbb8: Status 404 returned error can't find the container with id 80d941ea8ed2b680aff060980b7bb2b5789775e18070999d9f1775c60dbcbbb8
Sep 11 00:00:31.702481 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:31.702464    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 00:00:32.566042 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.566006    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" event={"ID":"d5c28768-6b40-4b1c-bf4d-119a4b9e6950","Type":"ContainerStarted","Data":"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"}
Sep 11 00:00:32.567733 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.567696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" event={"ID":"a08334e8-6218-4f8c-bb17-a21b1c881368","Type":"ContainerStarted","Data":"cb50fb40dcd23287daf10cd1c6a28311560884a6b83ef032e684af9966170716"}
Sep 11 00:00:32.567866 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.567741    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" event={"ID":"a08334e8-6218-4f8c-bb17-a21b1c881368","Type":"ContainerStarted","Data":"80d941ea8ed2b680aff060980b7bb2b5789775e18070999d9f1775c60dbcbbb8"}
Sep 11 00:00:32.567866 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.567775    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:32.590043 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.590003    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" podStartSLOduration=2.589989216 podStartE2EDuration="2.589989216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:00:32.587318149 +0000 UTC m=+1422.853315018" watchObservedRunningTime="2026-09-11 00:00:32.589989216 +0000 UTC m=+1422.855986085"
Sep 11 00:00:32.629066 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:32.629018    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" podStartSLOduration=0.936350665 podStartE2EDuration="1.629001714s" totalImagesPullingTime="692.651049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:00:32.624585048 +0000 UTC m=+1422.890581938" watchObservedRunningTime="2026-09-11 00:00:32.629001714 +0000 UTC m=+1422.894998584"
Sep 11 00:00:38.579191 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:38.579164    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:00:41.246912 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:41.246883    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:41.247269 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:41.246922    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:41.258969 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:41.258941    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:00:41.627431 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:00:41.627403    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:01:00.327081 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.327049    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:01:00.332151 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.332123    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.334444 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.334417    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5ee885-serving-cert\""
Sep 11 00:01:00.343542 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.343520    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:01:00.465829 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.465788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.466028 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.465836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf28g\" (UniqueName: \"kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.466028 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.465861    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.466028 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.465901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.566610 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.566579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.566792 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.566623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf28g\" (UniqueName: \"kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.566792 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.566646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.566792 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.566669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.567371 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.567340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.569573 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.569548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.569693 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.569588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.575821 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.575788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf28g\" (UniqueName: \"kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g\") pod \"maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.644881 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.644793    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:00.939801 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:01:00.939767    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda982dfce_748e_4081_8632_c513702358be.slice/crio-3f2fc903c91ab0a7a7d3753468acd36a534fb6e0cbded6eb60ec1ada2393df74 WatchSource:0}: Error finding container 3f2fc903c91ab0a7a7d3753468acd36a534fb6e0cbded6eb60ec1ada2393df74: Status 404 returned error can't find the container with id 3f2fc903c91ab0a7a7d3753468acd36a534fb6e0cbded6eb60ec1ada2393df74
Sep 11 00:01:00.941234 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:00.941195    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:01:01.730320 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:01.730230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" event={"ID":"a982dfce-748e-4081-8632-c513702358be","Type":"ContainerStarted","Data":"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"}
Sep 11 00:01:01.730320 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:01.730282    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" event={"ID":"a982dfce-748e-4081-8632-c513702358be","Type":"ContainerStarted","Data":"3f2fc903c91ab0a7a7d3753468acd36a534fb6e0cbded6eb60ec1ada2393df74"}
Sep 11 00:01:01.730765 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:01.730350    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:01.751627 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:01.751579    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" podStartSLOduration=1.256385265 podStartE2EDuration="1.751562907s" totalImagesPullingTime="495.177642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:01:01.749721894 +0000 UTC m=+1452.015718825" watchObservedRunningTime="2026-09-11 00:01:01.751562907 +0000 UTC m=+1452.017559776"
Sep 11 00:01:07.740955 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:07.740928    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:01:31.221089 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:31.221045    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:01:46.249269 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.249195    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" podUID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" containerName="main" containerID="cri-o://f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49" gracePeriod=45
Sep 11 00:01:46.508921 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.508865    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:01:46.613377 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613347    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqnlq\" (UniqueName: \"kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613417    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613462    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613521    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613550    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613773 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613568    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs\") pod \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\" (UID: \"d5c28768-6b40-4b1c-bf4d-119a4b9e6950\") "
Sep 11 00:01:46.613773 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613694    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:01:46.613773 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613709    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:01:46.613941 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613833    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home" (OuterVolumeSpecName: "home") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:01:46.613941 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613918    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-model-cache\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.614036 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613952    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tmp-dir\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.614036 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.613960    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-home\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.615759 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.615724    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm" (OuterVolumeSpecName: "dshm") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:01:46.615889 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.615790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:01:46.615889 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.615793    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq" (OuterVolumeSpecName: "kube-api-access-hqnlq") pod "d5c28768-6b40-4b1c-bf4d-119a4b9e6950" (UID: "d5c28768-6b40-4b1c-bf4d-119a4b9e6950"). InnerVolumeSpecName "kube-api-access-hqnlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:01:46.715408 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.715376    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-dshm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.715408 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.715404    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-tls-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.715408 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.715417    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqnlq\" (UniqueName: \"kubernetes.io/projected/d5c28768-6b40-4b1c-bf4d-119a4b9e6950-kube-api-access-hqnlq\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:46.971674 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.971638    2575 generic.go:356] "Generic (PLEG): container finished" podID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" containerID="f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49" exitCode=0
Sep 11 00:01:46.971861 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.971680    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"
Sep 11 00:01:46.971861 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.971724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" event={"ID":"d5c28768-6b40-4b1c-bf4d-119a4b9e6950","Type":"ContainerDied","Data":"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"}
Sep 11 00:01:46.971861 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.971772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk" event={"ID":"d5c28768-6b40-4b1c-bf4d-119a4b9e6950","Type":"ContainerDied","Data":"6611fa34a9f1a8d1f7ee11a5dcd13758eaa5479d2511ce14270064bc8be1a53b"}
Sep 11 00:01:46.971861 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.971794    2575 scope.go:117] "RemoveContainer" containerID="f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"
Sep 11 00:01:46.985334 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.985315    2575 scope.go:117] "RemoveContainer" containerID="f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"
Sep 11 00:01:46.985677 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:01:46.985660    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49\": container with ID starting with f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49 not found: ID does not exist" containerID="f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"
Sep 11 00:01:46.985736 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.985687    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49"} err="failed to get container status \"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49\": rpc error: code = NotFound desc = could not find container \"f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49\": container with ID starting with f13b257248d08b12e6ecc8940846f2224562d497bad5fc092fb44fee4bfbbd49 not found: ID does not exist"
Sep 11 00:01:46.995974 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.995950    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:01:46.999964 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:46.999943    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e96d4c/sub-test-model-e96d4c-kserve-7d974f6457-4tnsk"]
Sep 11 00:01:47.822169 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:47.822136    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:01:47.822616 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:47.822381    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" podUID="a08334e8-6218-4f8c-bb17-a21b1c881368" containerName="maas-api" containerID="cri-o://cb50fb40dcd23287daf10cd1c6a28311560884a6b83ef032e684af9966170716" gracePeriod=30
Sep 11 00:01:47.981870 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:47.981841    2575 generic.go:356] "Generic (PLEG): container finished" podID="a08334e8-6218-4f8c-bb17-a21b1c881368" containerID="cb50fb40dcd23287daf10cd1c6a28311560884a6b83ef032e684af9966170716" exitCode=0
Sep 11 00:01:47.982037 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:47.981914    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" event={"ID":"a08334e8-6218-4f8c-bb17-a21b1c881368","Type":"ContainerDied","Data":"cb50fb40dcd23287daf10cd1c6a28311560884a6b83ef032e684af9966170716"}
Sep 11 00:01:48.077406 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.077346    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:01:48.229102 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.229067    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" path="/var/lib/kubelet/pods/d5c28768-6b40-4b1c-bf4d-119a4b9e6950/volumes"
Sep 11 00:01:48.229721 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.229704    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls\") pod \"a08334e8-6218-4f8c-bb17-a21b1c881368\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") "
Sep 11 00:01:48.229782 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.229737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca\") pod \"a08334e8-6218-4f8c-bb17-a21b1c881368\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") "
Sep 11 00:01:48.229820 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.229811    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ndvp\" (UniqueName: \"kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp\") pod \"a08334e8-6218-4f8c-bb17-a21b1c881368\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") "
Sep 11 00:01:48.229881 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.229861    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs\") pod \"a08334e8-6218-4f8c-bb17-a21b1c881368\" (UID: \"a08334e8-6218-4f8c-bb17-a21b1c881368\") "
Sep 11 00:01:48.230224 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.230194    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a08334e8-6218-4f8c-bb17-a21b1c881368" (UID: "a08334e8-6218-4f8c-bb17-a21b1c881368"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:01:48.232149 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.232128    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a08334e8-6218-4f8c-bb17-a21b1c881368" (UID: "a08334e8-6218-4f8c-bb17-a21b1c881368"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:01:48.232600 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.232577    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a08334e8-6218-4f8c-bb17-a21b1c881368" (UID: "a08334e8-6218-4f8c-bb17-a21b1c881368"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:01:48.232707 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.232605    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp" (OuterVolumeSpecName: "kube-api-access-6ndvp") pod "a08334e8-6218-4f8c-bb17-a21b1c881368" (UID: "a08334e8-6218-4f8c-bb17-a21b1c881368"). InnerVolumeSpecName "kube-api-access-6ndvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:01:48.330961 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.330890    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:48.330961 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.330920    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a08334e8-6218-4f8c-bb17-a21b1c881368-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:48.330961 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.330929    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ndvp\" (UniqueName: \"kubernetes.io/projected/a08334e8-6218-4f8c-bb17-a21b1c881368-kube-api-access-6ndvp\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:48.330961 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.330940    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a08334e8-6218-4f8c-bb17-a21b1c881368-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:01:48.988328 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.988289    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw" event={"ID":"a08334e8-6218-4f8c-bb17-a21b1c881368","Type":"ContainerDied","Data":"80d941ea8ed2b680aff060980b7bb2b5789775e18070999d9f1775c60dbcbbb8"}
Sep 11 00:01:48.988328 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.988310    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"
Sep 11 00:01:48.988833 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:48.988349    2575 scope.go:117] "RemoveContainer" containerID="cb50fb40dcd23287daf10cd1c6a28311560884a6b83ef032e684af9966170716"
Sep 11 00:01:49.010585 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:49.010560    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:01:49.015671 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:49.015649    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aef6df-7f79b6bb4c-t2gcw"]
Sep 11 00:01:50.229762 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:01:50.229720    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a08334e8-6218-4f8c-bb17-a21b1c881368" path="/var/lib/kubelet/pods/a08334e8-6218-4f8c-bb17-a21b1c881368/volumes"
Sep 11 00:02:10.344522 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.344465    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:02:10.345133 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.344724    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" podUID="a982dfce-748e-4081-8632-c513702358be" containerName="maas-api" containerID="cri-o://56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64" gracePeriod=30
Sep 11 00:02:10.590509 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.590474    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:02:10.649819 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.649739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls\") pod \"a982dfce-748e-4081-8632-c513702358be\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") "
Sep 11 00:02:10.649819 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.649785    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca\") pod \"a982dfce-748e-4081-8632-c513702358be\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") "
Sep 11 00:02:10.649819 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.649808    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf28g\" (UniqueName: \"kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g\") pod \"a982dfce-748e-4081-8632-c513702358be\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") "
Sep 11 00:02:10.650119 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.649897    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs\") pod \"a982dfce-748e-4081-8632-c513702358be\" (UID: \"a982dfce-748e-4081-8632-c513702358be\") "
Sep 11 00:02:10.650201 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.650171    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a982dfce-748e-4081-8632-c513702358be" (UID: "a982dfce-748e-4081-8632-c513702358be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:10.650387 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.650372    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a982dfce-748e-4081-8632-c513702358be-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.652049 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.652028    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a982dfce-748e-4081-8632-c513702358be" (UID: "a982dfce-748e-4081-8632-c513702358be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:10.652127 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.652084    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g" (OuterVolumeSpecName: "kube-api-access-jf28g") pod "a982dfce-748e-4081-8632-c513702358be" (UID: "a982dfce-748e-4081-8632-c513702358be"). InnerVolumeSpecName "kube-api-access-jf28g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:10.652181 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.652123    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a982dfce-748e-4081-8632-c513702358be" (UID: "a982dfce-748e-4081-8632-c513702358be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:10.751704 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.751671    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.751704 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.751701    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf28g\" (UniqueName: \"kubernetes.io/projected/a982dfce-748e-4081-8632-c513702358be-kube-api-access-jf28g\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.751913 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:10.751715    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a982dfce-748e-4081-8632-c513702358be-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:11.112448 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.112410    2575 generic.go:356] "Generic (PLEG): container finished" podID="a982dfce-748e-4081-8632-c513702358be" containerID="56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64" exitCode=0
Sep 11 00:02:11.112632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.112455    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"
Sep 11 00:02:11.112632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.112521    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" event={"ID":"a982dfce-748e-4081-8632-c513702358be","Type":"ContainerDied","Data":"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"}
Sep 11 00:02:11.112632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.112572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m" event={"ID":"a982dfce-748e-4081-8632-c513702358be","Type":"ContainerDied","Data":"3f2fc903c91ab0a7a7d3753468acd36a534fb6e0cbded6eb60ec1ada2393df74"}
Sep 11 00:02:11.112632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.112596    2575 scope.go:117] "RemoveContainer" containerID="56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"
Sep 11 00:02:11.122665 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.122649    2575 scope.go:117] "RemoveContainer" containerID="56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"
Sep 11 00:02:11.122935 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:02:11.122915    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64\": container with ID starting with 56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64 not found: ID does not exist" containerID="56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"
Sep 11 00:02:11.122989 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.122944    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64"} err="failed to get container status \"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64\": rpc error: code = NotFound desc = could not find container \"56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64\": container with ID starting with 56b81ed4d2a5e29e0863bdc6ae50de1a8d1874073ccdf61f98dd728ec1135c64 not found: ID does not exist"
Sep 11 00:02:11.136213 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.136185    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:02:11.139942 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:11.139920    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ee885-5b98ff589d-nsq6m"]
Sep 11 00:02:12.228635 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:12.228603    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a982dfce-748e-4081-8632-c513702358be" path="/var/lib/kubelet/pods/a982dfce-748e-4081-8632-c513702358be/volumes"
Sep 11 00:02:27.974036 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:27.974004    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 11 00:02:27.974536 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:27.974261    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" containerName="maas-api" containerID="cri-o://e675fbca5b4e5e07a32aed63dfbf09c2e18da655817a2214e0adcebfd6e0781b" gracePeriod=30
Sep 11 00:02:28.031358 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.031321    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.57:8443/health\": dial tcp 10.134.0.57:8443: connect: connection refused"
Sep 11 00:02:28.205444 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.205412    2575 generic.go:356] "Generic (PLEG): container finished" podID="08520636-f725-4d5f-8d71-e8e8578395dd" containerID="e675fbca5b4e5e07a32aed63dfbf09c2e18da655817a2214e0adcebfd6e0781b" exitCode=0
Sep 11 00:02:28.205640 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.205518    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" event={"ID":"08520636-f725-4d5f-8d71-e8e8578395dd","Type":"ContainerDied","Data":"e675fbca5b4e5e07a32aed63dfbf09c2e18da655817a2214e0adcebfd6e0781b"}
Sep 11 00:02:28.233543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.233465    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 11 00:02:28.320186 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls\") pod \"08520636-f725-4d5f-8d71-e8e8578395dd\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") "
Sep 11 00:02:28.320349 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320238    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs\") pod \"08520636-f725-4d5f-8d71-e8e8578395dd\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") "
Sep 11 00:02:28.320349 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320263    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca\") pod \"08520636-f725-4d5f-8d71-e8e8578395dd\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") "
Sep 11 00:02:28.320349 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ct87\" (UniqueName: \"kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87\") pod \"08520636-f725-4d5f-8d71-e8e8578395dd\" (UID: \"08520636-f725-4d5f-8d71-e8e8578395dd\") "
Sep 11 00:02:28.320687 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320661    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08520636-f725-4d5f-8d71-e8e8578395dd" (UID: "08520636-f725-4d5f-8d71-e8e8578395dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:28.320807 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.320758    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08520636-f725-4d5f-8d71-e8e8578395dd-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.322238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.322206    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08520636-f725-4d5f-8d71-e8e8578395dd" (UID: "08520636-f725-4d5f-8d71-e8e8578395dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:28.322366 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.322348    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "08520636-f725-4d5f-8d71-e8e8578395dd" (UID: "08520636-f725-4d5f-8d71-e8e8578395dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:28.322609 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.322589    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87" (OuterVolumeSpecName: "kube-api-access-6ct87") pod "08520636-f725-4d5f-8d71-e8e8578395dd" (UID: "08520636-f725-4d5f-8d71-e8e8578395dd"). InnerVolumeSpecName "kube-api-access-6ct87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:28.422241 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.422213    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.422241 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.422238    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08520636-f725-4d5f-8d71-e8e8578395dd-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.422241 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:28.422247    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ct87\" (UniqueName: \"kubernetes.io/projected/08520636-f725-4d5f-8d71-e8e8578395dd-kube-api-access-6ct87\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:29.211902 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:29.211867    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj" event={"ID":"08520636-f725-4d5f-8d71-e8e8578395dd","Type":"ContainerDied","Data":"df60db790000abb531475f1e1dbb3b9cb2846dc44f69fef20f9d1ade7d2441da"}
Sep 11 00:02:29.212381 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:29.211953    2575 scope.go:117] "RemoveContainer" containerID="e675fbca5b4e5e07a32aed63dfbf09c2e18da655817a2214e0adcebfd6e0781b"
Sep 11 00:02:29.212381 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:29.211883    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"
Sep 11 00:02:29.253658 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:29.253636    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 11 00:02:29.264660 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:29.264636    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9eee5d-6d5887845c-bw6lj"]
Sep 11 00:02:30.228194 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:30.228160    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" path="/var/lib/kubelet/pods/08520636-f725-4d5f-8d71-e8e8578395dd/volumes"
Sep 11 00:02:45.368018 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.367985    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 11 00:02:45.368640 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.368244    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" podUID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" containerName="maas-api" containerID="cri-o://28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909" gracePeriod=30
Sep 11 00:02:45.618557 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.618502    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 11 00:02:45.680918 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.680886    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca\") pod \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") "
Sep 11 00:02:45.681064 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.680947    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk5p4\" (UniqueName: \"kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4\") pod \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") "
Sep 11 00:02:45.681064 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.681016    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs\") pod \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") "
Sep 11 00:02:45.681064 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.681037    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls\") pod \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\" (UID: \"aa69bef5-cf8d-49b2-9bc6-877a15826a4d\") "
Sep 11 00:02:45.681267 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.681247    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa69bef5-cf8d-49b2-9bc6-877a15826a4d" (UID: "aa69bef5-cf8d-49b2-9bc6-877a15826a4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:45.681430 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.681416    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:45.683217 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.683196    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa69bef5-cf8d-49b2-9bc6-877a15826a4d" (UID: "aa69bef5-cf8d-49b2-9bc6-877a15826a4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:45.683645 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.683630    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4" (OuterVolumeSpecName: "kube-api-access-sk5p4") pod "aa69bef5-cf8d-49b2-9bc6-877a15826a4d" (UID: "aa69bef5-cf8d-49b2-9bc6-877a15826a4d"). InnerVolumeSpecName "kube-api-access-sk5p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:45.683645 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.683637    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa69bef5-cf8d-49b2-9bc6-877a15826a4d" (UID: "aa69bef5-cf8d-49b2-9bc6-877a15826a4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:02:45.782604 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.782575    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:45.782604 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.782598    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:45.782604 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:45.782607    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk5p4\" (UniqueName: \"kubernetes.io/projected/aa69bef5-cf8d-49b2-9bc6-877a15826a4d-kube-api-access-sk5p4\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:02:46.299516 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.299462    2575 generic.go:356] "Generic (PLEG): container finished" podID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" containerID="28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909" exitCode=0
Sep 11 00:02:46.299667 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.299531    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"
Sep 11 00:02:46.299667 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.299542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" event={"ID":"aa69bef5-cf8d-49b2-9bc6-877a15826a4d","Type":"ContainerDied","Data":"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"}
Sep 11 00:02:46.299667 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.299595    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7" event={"ID":"aa69bef5-cf8d-49b2-9bc6-877a15826a4d","Type":"ContainerDied","Data":"1539f4c61081e7fff2f648b47041ae43f7ffa032907216cc1dd3c656ea2fa91c"}
Sep 11 00:02:46.299667 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.299618    2575 scope.go:117] "RemoveContainer" containerID="28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"
Sep 11 00:02:46.308884 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.308865    2575 scope.go:117] "RemoveContainer" containerID="28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"
Sep 11 00:02:46.309157 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:02:46.309140    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909\": container with ID starting with 28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909 not found: ID does not exist" containerID="28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"
Sep 11 00:02:46.309230 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.309165    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909"} err="failed to get container status \"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909\": rpc error: code = NotFound desc = could not find container \"28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909\": container with ID starting with 28738a00456cfb102289aa4516534380bf83c81ea19e7f19dd50e0866b388909 not found: ID does not exist"
Sep 11 00:02:46.315160 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.315135    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 11 00:02:46.318898 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:46.318876    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dd5958-599d7f4c6d-tbsl7"]
Sep 11 00:02:48.228128 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:02:48.228083    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" path="/var/lib/kubelet/pods/aa69bef5-cf8d-49b2-9bc6-877a15826a4d/volumes"
Sep 11 00:03:14.370058 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370018    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:03:14.370575 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370535    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" containerName="maas-api"
Sep 11 00:03:14.370575 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370550    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" containerName="maas-api"
Sep 11 00:03:14.370575 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370575    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a08334e8-6218-4f8c-bb17-a21b1c881368" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370581    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a08334e8-6218-4f8c-bb17-a21b1c881368" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370599    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a982dfce-748e-4081-8632-c513702358be" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370605    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a982dfce-748e-4081-8632-c513702358be" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370615    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370620    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370626    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" containerName="main"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370631    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" containerName="main"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370698    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a982dfce-748e-4081-8632-c513702358be" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370708    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a08334e8-6218-4f8c-bb17-a21b1c881368" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370718    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c28768-6b40-4b1c-bf4d-119a4b9e6950" containerName="main"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370727    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="08520636-f725-4d5f-8d71-e8e8578395dd" containerName="maas-api"
Sep 11 00:03:14.370749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.370732    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa69bef5-cf8d-49b2-9bc6-877a15826a4d" containerName="maas-api"
Sep 11 00:03:14.375167 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.375144    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.377323 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.377301    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-51d6fb-serving-cert\""
Sep 11 00:03:14.384413 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.383989    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:03:14.555886 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.555853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.556093 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.555941    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.556093 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.555972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzst5\" (UniqueName: \"kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.556093 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.556006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.657007 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.656930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.657007 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.656980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzst5\" (UniqueName: \"kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.657225 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.657013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.657225 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.657048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.657225 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:03:14.657069    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-serving-cert: secret "maas-api-e2e-ipp-a-51d6fb-serving-cert" not found
Sep 11 00:03:14.657225 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:03:14.657140    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls podName:ae1164d7-52b0-420f-b114-7fe3a84afad3 nodeName:}" failed. No retries permitted until 2026-09-11 00:03:15.157120645 +0000 UTC m=+1585.423117494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls") pod "maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" (UID: "ae1164d7-52b0-420f-b114-7fe3a84afad3") : secret "maas-api-e2e-ipp-a-51d6fb-serving-cert" not found
Sep 11 00:03:14.657720 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.657701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.659616 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.659596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:14.665225 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:14.665193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzst5\" (UniqueName: \"kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:15.161870 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:15.161837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:15.164480 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:15.164459    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:15.288455 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:15.288420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:15.562244 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:15.562219    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:03:15.564951 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:03:15.564920    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1164d7_52b0_420f_b114_7fe3a84afad3.slice/crio-b28a9af2263069c9520a0be9003b74b4d811cb8b70b98799a9469ee7c8d7fb07 WatchSource:0}: Error finding container b28a9af2263069c9520a0be9003b74b4d811cb8b70b98799a9469ee7c8d7fb07: Status 404 returned error can't find the container with id b28a9af2263069c9520a0be9003b74b4d811cb8b70b98799a9469ee7c8d7fb07
Sep 11 00:03:16.454796 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:16.454702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" event={"ID":"ae1164d7-52b0-420f-b114-7fe3a84afad3","Type":"ContainerStarted","Data":"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"}
Sep 11 00:03:16.454796 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:16.454760    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:16.454796 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:16.454776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" event={"ID":"ae1164d7-52b0-420f-b114-7fe3a84afad3","Type":"ContainerStarted","Data":"b28a9af2263069c9520a0be9003b74b4d811cb8b70b98799a9469ee7c8d7fb07"}
Sep 11 00:03:16.471662 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:16.471618    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" podStartSLOduration=1.970856122 podStartE2EDuration="2.471603164s" totalImagesPullingTime="500.747042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:03:16.470833126 +0000 UTC m=+1586.736830036" watchObservedRunningTime="2026-09-11 00:03:16.471603164 +0000 UTC m=+1586.737600035"
Sep 11 00:03:22.466190 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:22.466153    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:03:43.547854 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.547756    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:03:43.551630 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.551608    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.553833 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.553805    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c416fa-serving-cert\""
Sep 11 00:03:43.560836 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.560814    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:03:43.633958 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.633924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.634123 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.633968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.634123 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.633997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.634123 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.634062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgds6\" (UniqueName: \"kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.735359 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.735331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.735537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.735367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgds6\" (UniqueName: \"kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.735537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.735517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.735661 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.735553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.735719 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:03:43.735672    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-serving-cert: secret "maas-api-e2e-ipp-b-c416fa-serving-cert" not found
Sep 11 00:03:43.735772 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:03:43.735755    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls podName:1cb7fc59-bf3b-4549-9e85-a3025455181c nodeName:}" failed. No retries permitted until 2026-09-11 00:03:44.235733864 +0000 UTC m=+1614.501730731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls") pod "maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" (UID: "1cb7fc59-bf3b-4549-9e85-a3025455181c") : secret "maas-api-e2e-ipp-b-c416fa-serving-cert" not found
Sep 11 00:03:43.736118 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.736096    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.738108 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.738089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:43.746045 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:43.746020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgds6\" (UniqueName: \"kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:44.241036 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:44.241002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:44.243798 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:44.243762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:44.464988 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:44.464914    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:44.739210 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:44.739182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:03:44.740714 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:03:44.740677    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb7fc59_bf3b_4549_9e85_a3025455181c.slice/crio-7f976b28dddc94d9a561afb047683e90e83bb920347bbdd43c4cc8452cbe47bf WatchSource:0}: Error finding container 7f976b28dddc94d9a561afb047683e90e83bb920347bbdd43c4cc8452cbe47bf: Status 404 returned error can't find the container with id 7f976b28dddc94d9a561afb047683e90e83bb920347bbdd43c4cc8452cbe47bf
Sep 11 00:03:45.617132 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:45.617093    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" event={"ID":"1cb7fc59-bf3b-4549-9e85-a3025455181c","Type":"ContainerStarted","Data":"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"}
Sep 11 00:03:45.617334 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:45.617158    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" event={"ID":"1cb7fc59-bf3b-4549-9e85-a3025455181c","Type":"ContainerStarted","Data":"7f976b28dddc94d9a561afb047683e90e83bb920347bbdd43c4cc8452cbe47bf"}
Sep 11 00:03:45.617334 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:45.617190    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:03:45.635042 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:45.634990    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" podStartSLOduration=2.149798258 podStartE2EDuration="2.634974628s" totalImagesPullingTime="485.17637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:03:45.633818218 +0000 UTC m=+1615.899815114" watchObservedRunningTime="2026-09-11 00:03:45.634974628 +0000 UTC m=+1615.900971498"
Sep 11 00:03:51.627835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:03:51.627789    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:04:12.260297 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:12.260272    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:04:53.148674 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:53.148640    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:04:57.900591 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.900556    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:04:57.904666 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.904644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:57.907044 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.907022    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-da536f-serving-cert\""
Sep 11 00:04:57.916708 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.916680    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:04:57.962818 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.962780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:57.962818 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.962821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:57.963049 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.962874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:57.963049 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:57.962985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4f9j\" (UniqueName: \"kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.064050 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.064009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.064050 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.064052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.064303 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.064091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.064303 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:04:58.064177    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-serving-cert: secret "maas-api-e2e-ipp-cleanup-da536f-serving-cert" not found
Sep 11 00:04:58.064303 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.064211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4f9j\" (UniqueName: \"kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.064303 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:04:58.064253    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls podName:a0efc0ab-6b16-4db2-a4ee-fec3a78d2633 nodeName:}" failed. No retries permitted until 2026-09-11 00:04:58.564234371 +0000 UTC m=+1688.830231220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" (UID: "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633") : secret "maas-api-e2e-ipp-cleanup-da536f-serving-cert" not found
Sep 11 00:04:58.064840 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.064821    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.066726 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.066708    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.075858 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.075822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4f9j\" (UniqueName: \"kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.569185 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.569142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.571794 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.571772    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:58.820485 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:58.820400    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:04:59.096710 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:04:59.096675    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:04:59.099103 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:04:59.099078    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0efc0ab_6b16_4db2_a4ee_fec3a78d2633.slice/crio-bb23edaa3f1337a4dbec27083b0669c4ab249dea5a7d84cd778eb050407903a7 WatchSource:0}: Error finding container bb23edaa3f1337a4dbec27083b0669c4ab249dea5a7d84cd778eb050407903a7: Status 404 returned error can't find the container with id bb23edaa3f1337a4dbec27083b0669c4ab249dea5a7d84cd778eb050407903a7
Sep 11 00:05:00.009300 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:00.009268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" event={"ID":"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633","Type":"ContainerStarted","Data":"bb23edaa3f1337a4dbec27083b0669c4ab249dea5a7d84cd778eb050407903a7"}
Sep 11 00:05:01.016464 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:01.016424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" event={"ID":"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633","Type":"ContainerStarted","Data":"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"}
Sep 11 00:05:01.016880 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:01.016604    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:05:01.033540 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:01.033464    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" podStartSLOduration=3.174181656 podStartE2EDuration="4.033445898s" totalImagesPullingTime="859.264242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:05:01.031751129 +0000 UTC m=+1691.297748001" watchObservedRunningTime="2026-09-11 00:05:01.033445898 +0000 UTC m=+1691.299442769"
Sep 11 00:05:07.027477 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:07.027435    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:05:28.475088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.474996    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:05:28.475669 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.475241    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" podUID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" containerName="maas-api" containerID="cri-o://0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a" gracePeriod=30
Sep 11 00:05:28.729577 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.729471    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:05:28.777968 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.777938    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca\") pod \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") "
Sep 11 00:05:28.778173 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.777983    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs\") pod \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") "
Sep 11 00:05:28.778173 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.778037    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4f9j\" (UniqueName: \"kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j\") pod \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") "
Sep 11 00:05:28.778173 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.778168    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") pod \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\" (UID: \"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633\") "
Sep 11 00:05:28.778429 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.778385    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" (UID: "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:05:28.778609 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.778531    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:28.780407 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.780384    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" (UID: "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:05:28.780407 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.780390    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j" (OuterVolumeSpecName: "kube-api-access-r4f9j") pod "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" (UID: "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633"). InnerVolumeSpecName "kube-api-access-r4f9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:05:28.780573 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.780454    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" (UID: "a0efc0ab-6b16-4db2-a4ee-fec3a78d2633"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:05:28.879250 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.879205    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:28.879250 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.879240    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:28.879250 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:28.879249    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4f9j\" (UniqueName: \"kubernetes.io/projected/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633-kube-api-access-r4f9j\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:29.173236 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.173197    2575 generic.go:356] "Generic (PLEG): container finished" podID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" containerID="0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a" exitCode=0
Sep 11 00:05:29.173236 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.173240    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"
Sep 11 00:05:29.173462 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.173283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" event={"ID":"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633","Type":"ContainerDied","Data":"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"}
Sep 11 00:05:29.173462 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.173340    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95" event={"ID":"a0efc0ab-6b16-4db2-a4ee-fec3a78d2633","Type":"ContainerDied","Data":"bb23edaa3f1337a4dbec27083b0669c4ab249dea5a7d84cd778eb050407903a7"}
Sep 11 00:05:29.173462 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.173370    2575 scope.go:117] "RemoveContainer" containerID="0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"
Sep 11 00:05:29.183332 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.183314    2575 scope.go:117] "RemoveContainer" containerID="0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"
Sep 11 00:05:29.183665 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:05:29.183645    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a\": container with ID starting with 0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a not found: ID does not exist" containerID="0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"
Sep 11 00:05:29.183718 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.183675    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a"} err="failed to get container status \"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a\": rpc error: code = NotFound desc = could not find container \"0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a\": container with ID starting with 0e95d6df87c4c2854bb58808f99a6b93aac5c47b99c4601b5d24b853d582502a not found: ID does not exist"
Sep 11 00:05:29.194356 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.194328    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:05:29.200064 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:29.200039    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-da536f-58c9fd8c74-chj95"]
Sep 11 00:05:30.236482 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:30.236441    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" path="/var/lib/kubelet/pods/a0efc0ab-6b16-4db2-a4ee-fec3a78d2633/volumes"
Sep 11 00:05:54.001348 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.001308    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:05:54.001853 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.001628    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" podUID="ae1164d7-52b0-420f-b114-7fe3a84afad3" containerName="maas-api" containerID="cri-o://70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7" gracePeriod=30
Sep 11 00:05:54.260810 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.260747    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:05:54.312104 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.312068    2575 generic.go:356] "Generic (PLEG): container finished" podID="ae1164d7-52b0-420f-b114-7fe3a84afad3" containerID="70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7" exitCode=0
Sep 11 00:05:54.312289 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.312116    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"
Sep 11 00:05:54.312289 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.312155    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" event={"ID":"ae1164d7-52b0-420f-b114-7fe3a84afad3","Type":"ContainerDied","Data":"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"}
Sep 11 00:05:54.312289 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.312211    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww" event={"ID":"ae1164d7-52b0-420f-b114-7fe3a84afad3","Type":"ContainerDied","Data":"b28a9af2263069c9520a0be9003b74b4d811cb8b70b98799a9469ee7c8d7fb07"}
Sep 11 00:05:54.312289 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.312241    2575 scope.go:117] "RemoveContainer" containerID="70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"
Sep 11 00:05:54.321694 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.321677    2575 scope.go:117] "RemoveContainer" containerID="70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"
Sep 11 00:05:54.321983 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:05:54.321963    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7\": container with ID starting with 70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7 not found: ID does not exist" containerID="70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"
Sep 11 00:05:54.322052 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.321998    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7"} err="failed to get container status \"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7\": rpc error: code = NotFound desc = could not find container \"70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7\": container with ID starting with 70c04dcdd021d13e1fa6a8527567a86234f9e2dff59045be1f9742f5065be9c7 not found: ID does not exist"
Sep 11 00:05:54.348556 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.348529    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzst5\" (UniqueName: \"kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5\") pod \"ae1164d7-52b0-420f-b114-7fe3a84afad3\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") "
Sep 11 00:05:54.348744 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.348574    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") pod \"ae1164d7-52b0-420f-b114-7fe3a84afad3\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") "
Sep 11 00:05:54.348744 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.348618    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca\") pod \"ae1164d7-52b0-420f-b114-7fe3a84afad3\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") "
Sep 11 00:05:54.348869 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.348772    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs\") pod \"ae1164d7-52b0-420f-b114-7fe3a84afad3\" (UID: \"ae1164d7-52b0-420f-b114-7fe3a84afad3\") "
Sep 11 00:05:54.348948 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.348923    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae1164d7-52b0-420f-b114-7fe3a84afad3" (UID: "ae1164d7-52b0-420f-b114-7fe3a84afad3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:05:54.349213 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.349196    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1164d7-52b0-420f-b114-7fe3a84afad3-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:54.350822 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.350800    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae1164d7-52b0-420f-b114-7fe3a84afad3" (UID: "ae1164d7-52b0-420f-b114-7fe3a84afad3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:05:54.350954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.350931    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae1164d7-52b0-420f-b114-7fe3a84afad3" (UID: "ae1164d7-52b0-420f-b114-7fe3a84afad3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:05:54.350954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.350943    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5" (OuterVolumeSpecName: "kube-api-access-gzst5") pod "ae1164d7-52b0-420f-b114-7fe3a84afad3" (UID: "ae1164d7-52b0-420f-b114-7fe3a84afad3"). InnerVolumeSpecName "kube-api-access-gzst5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:05:54.450481 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.450426    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzst5\" (UniqueName: \"kubernetes.io/projected/ae1164d7-52b0-420f-b114-7fe3a84afad3-kube-api-access-gzst5\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:54.450481 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.450475    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:54.450481 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.450485    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae1164d7-52b0-420f-b114-7fe3a84afad3-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:05:54.634435 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.634401    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:05:54.638421 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:54.638387    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-51d6fb-79ff99fb8d-gp7ww"]
Sep 11 00:05:56.228659 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:05:56.228621    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1164d7-52b0-420f-b114-7fe3a84afad3" path="/var/lib/kubelet/pods/ae1164d7-52b0-420f-b114-7fe3a84afad3/volumes"
Sep 11 00:07:24.972373 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:24.972335    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:07:24.972998 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:24.972592    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" podUID="1cb7fc59-bf3b-4549-9e85-a3025455181c" containerName="maas-api" containerID="cri-o://ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415" gracePeriod=30
Sep 11 00:07:25.228701 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.228636    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:07:25.324931 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.324900    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") pod \"1cb7fc59-bf3b-4549-9e85-a3025455181c\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") "
Sep 11 00:07:25.325154 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.324961    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgds6\" (UniqueName: \"kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6\") pod \"1cb7fc59-bf3b-4549-9e85-a3025455181c\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") "
Sep 11 00:07:25.325154 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.325064    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs\") pod \"1cb7fc59-bf3b-4549-9e85-a3025455181c\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") "
Sep 11 00:07:25.325154 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.325096    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca\") pod \"1cb7fc59-bf3b-4549-9e85-a3025455181c\" (UID: \"1cb7fc59-bf3b-4549-9e85-a3025455181c\") "
Sep 11 00:07:25.326384 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.326354    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cb7fc59-bf3b-4549-9e85-a3025455181c" (UID: "1cb7fc59-bf3b-4549-9e85-a3025455181c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:07:25.327855 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.327776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cb7fc59-bf3b-4549-9e85-a3025455181c" (UID: "1cb7fc59-bf3b-4549-9e85-a3025455181c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:07:25.328011 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.327988    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6" (OuterVolumeSpecName: "kube-api-access-vgds6") pod "1cb7fc59-bf3b-4549-9e85-a3025455181c" (UID: "1cb7fc59-bf3b-4549-9e85-a3025455181c"). InnerVolumeSpecName "kube-api-access-vgds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:07:25.328232 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.328208    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1cb7fc59-bf3b-4549-9e85-a3025455181c" (UID: "1cb7fc59-bf3b-4549-9e85-a3025455181c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:07:25.427127 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.427089    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgds6\" (UniqueName: \"kubernetes.io/projected/1cb7fc59-bf3b-4549-9e85-a3025455181c-kube-api-access-vgds6\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:25.427127 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.427120    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:25.427127 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.427129    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb7fc59-bf3b-4549-9e85-a3025455181c-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:25.427127 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.427138    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb7fc59-bf3b-4549-9e85-a3025455181c-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:25.816645 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.816614    2575 generic.go:356] "Generic (PLEG): container finished" podID="1cb7fc59-bf3b-4549-9e85-a3025455181c" containerID="ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415" exitCode=0
Sep 11 00:07:25.816835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.816652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" event={"ID":"1cb7fc59-bf3b-4549-9e85-a3025455181c","Type":"ContainerDied","Data":"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"}
Sep 11 00:07:25.816835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.816681    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"
Sep 11 00:07:25.816835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.816716    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn" event={"ID":"1cb7fc59-bf3b-4549-9e85-a3025455181c","Type":"ContainerDied","Data":"7f976b28dddc94d9a561afb047683e90e83bb920347bbdd43c4cc8452cbe47bf"}
Sep 11 00:07:25.816835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.816738    2575 scope.go:117] "RemoveContainer" containerID="ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"
Sep 11 00:07:25.827047 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.827022    2575 scope.go:117] "RemoveContainer" containerID="ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"
Sep 11 00:07:25.827414 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:07:25.827385    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415\": container with ID starting with ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415 not found: ID does not exist" containerID="ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"
Sep 11 00:07:25.827535 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.827421    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415"} err="failed to get container status \"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415\": rpc error: code = NotFound desc = could not find container \"ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415\": container with ID starting with ec50d77f644484b10365a5f634b44ef24533cdbfc5e1be6292dea834ca235415 not found: ID does not exist"
Sep 11 00:07:25.853282 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.853250    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:07:25.871311 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:25.871282    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c416fa-649649c8fd-w6kzn"]
Sep 11 00:07:26.228749 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:26.228670    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cb7fc59-bf3b-4549-9e85-a3025455181c" path="/var/lib/kubelet/pods/1cb7fc59-bf3b-4549-9e85-a3025455181c/volumes"
Sep 11 00:07:42.577142 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.577105    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 11 00:07:42.577584 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.577317    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" podUID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" containerName="maas-api" containerID="cri-o://0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7" gracePeriod=30
Sep 11 00:07:42.844040 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.843963    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 11 00:07:42.913283 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.913244    2575 generic.go:356] "Generic (PLEG): container finished" podID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" containerID="0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7" exitCode=0
Sep 11 00:07:42.913522 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.913356    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" event={"ID":"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d","Type":"ContainerDied","Data":"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"}
Sep 11 00:07:42.913522 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.913470    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg" event={"ID":"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d","Type":"ContainerDied","Data":"b08666dd063d1b3e0494de3ed9a951e1b7c800f6756778df53b19e01d81e38b2"}
Sep 11 00:07:42.913522 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.913511    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"
Sep 11 00:07:42.913733 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.913537    2575 scope.go:117] "RemoveContainer" containerID="0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"
Sep 11 00:07:42.926195 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.926168    2575 scope.go:117] "RemoveContainer" containerID="0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"
Sep 11 00:07:42.926664 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:07:42.926631    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7\": container with ID starting with 0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7 not found: ID does not exist" containerID="0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"
Sep 11 00:07:42.926736 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:42.926680    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7"} err="failed to get container status \"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7\": rpc error: code = NotFound desc = could not find container \"0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7\": container with ID starting with 0bf0f7c39d6caece9137718cce0d19d93cafbb132f9d100d30db9e7c57608ad7 not found: ID does not exist"
Sep 11 00:07:43.006997 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.006960    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpbn6\" (UniqueName: \"kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6\") pod \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") "
Sep 11 00:07:43.007202 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.007027    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls\") pod \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") "
Sep 11 00:07:43.007202 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.007055    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs\") pod \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") "
Sep 11 00:07:43.007202 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.007095    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca\") pod \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\" (UID: \"6a2d1537-3398-4152-b3ef-79e5bdb8aa9d\") "
Sep 11 00:07:43.007608 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.007567    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" (UID: "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:07:43.009566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.009539    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" (UID: "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:07:43.009693 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.009666    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" (UID: "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:07:43.009738 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.009670    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6" (OuterVolumeSpecName: "kube-api-access-wpbn6") pod "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" (UID: "6a2d1537-3398-4152-b3ef-79e5bdb8aa9d"). InnerVolumeSpecName "kube-api-access-wpbn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:07:43.108482 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.108395    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpbn6\" (UniqueName: \"kubernetes.io/projected/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-kube-api-access-wpbn6\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:43.108482 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.108428    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:43.108482 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.108438    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:43.108482 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.108445    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:07:43.238284 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.238244    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 11 00:07:43.242692 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:43.242664    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-37692b-6c4d67878d-d76sg"]
Sep 11 00:07:44.229166 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:07:44.229132    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" path="/var/lib/kubelet/pods/6a2d1537-3398-4152-b3ef-79e5bdb8aa9d/volumes"
Sep 11 00:08:00.392951 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.392908    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 11 00:08:00.393827 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.393764    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" podUID="da110c99-f0a9-463c-8351-c2ec9f59ad15" containerName="maas-api" containerID="cri-o://c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950" gracePeriod=30
Sep 11 00:08:00.650509 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.650424    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 11 00:08:00.688373 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.688327    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs\") pod \"da110c99-f0a9-463c-8351-c2ec9f59ad15\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") "
Sep 11 00:08:00.688633 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.688384    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjpp2\" (UniqueName: \"kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2\") pod \"da110c99-f0a9-463c-8351-c2ec9f59ad15\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") "
Sep 11 00:08:00.688633 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.688424    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") pod \"da110c99-f0a9-463c-8351-c2ec9f59ad15\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") "
Sep 11 00:08:00.688633 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.688473    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca\") pod \"da110c99-f0a9-463c-8351-c2ec9f59ad15\" (UID: \"da110c99-f0a9-463c-8351-c2ec9f59ad15\") "
Sep 11 00:08:00.688993 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.688954    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da110c99-f0a9-463c-8351-c2ec9f59ad15" (UID: "da110c99-f0a9-463c-8351-c2ec9f59ad15"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:08:00.690920 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.690888    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "da110c99-f0a9-463c-8351-c2ec9f59ad15" (UID: "da110c99-f0a9-463c-8351-c2ec9f59ad15"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:00.690920 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.690899    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2" (OuterVolumeSpecName: "kube-api-access-hjpp2") pod "da110c99-f0a9-463c-8351-c2ec9f59ad15" (UID: "da110c99-f0a9-463c-8351-c2ec9f59ad15"). InnerVolumeSpecName "kube-api-access-hjpp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:08:00.691240 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.691216    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da110c99-f0a9-463c-8351-c2ec9f59ad15" (UID: "da110c99-f0a9-463c-8351-c2ec9f59ad15"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:00.789543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.789477    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.789543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.789534    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjpp2\" (UniqueName: \"kubernetes.io/projected/da110c99-f0a9-463c-8351-c2ec9f59ad15-kube-api-access-hjpp2\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.789543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.789545    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da110c99-f0a9-463c-8351-c2ec9f59ad15-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.789543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:00.789554    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da110c99-f0a9-463c-8351-c2ec9f59ad15-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:01.017681 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.017582    2575 generic.go:356] "Generic (PLEG): container finished" podID="da110c99-f0a9-463c-8351-c2ec9f59ad15" containerID="c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950" exitCode=0
Sep 11 00:08:01.017681 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.017632    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"
Sep 11 00:08:01.017900 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.017677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" event={"ID":"da110c99-f0a9-463c-8351-c2ec9f59ad15","Type":"ContainerDied","Data":"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"}
Sep 11 00:08:01.017900 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.017729    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf" event={"ID":"da110c99-f0a9-463c-8351-c2ec9f59ad15","Type":"ContainerDied","Data":"11ed70d6681c74e42e6fb935b20fd3ee0795edb3535fb62a579e2271a56364ee"}
Sep 11 00:08:01.017900 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.017751    2575 scope.go:117] "RemoveContainer" containerID="c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"
Sep 11 00:08:01.028449 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.028427    2575 scope.go:117] "RemoveContainer" containerID="c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"
Sep 11 00:08:01.028862 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:08:01.028840    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950\": container with ID starting with c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950 not found: ID does not exist" containerID="c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"
Sep 11 00:08:01.028925 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.028873    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950"} err="failed to get container status \"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950\": rpc error: code = NotFound desc = could not find container \"c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950\": container with ID starting with c4096764df16b2f1ece4467692aee74ec4bfb2650392920e512919f1cff40950 not found: ID does not exist"
Sep 11 00:08:01.041211 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.041175    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 11 00:08:01.045088 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:01.045060    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e96d4c-68cf9f5bcc-znfvf"]
Sep 11 00:08:02.228073 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:02.228036    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da110c99-f0a9-463c-8351-c2ec9f59ad15" path="/var/lib/kubelet/pods/da110c99-f0a9-463c-8351-c2ec9f59ad15/volumes"
Sep 11 00:08:09.750702 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:09.750664    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 11 00:08:09.754515 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:09.751344    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" podUID="5832202a-b15c-4121-be80-6044399ed222" containerName="manager" containerID="cri-o://8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058" gracePeriod=30
Sep 11 00:08:10.019252 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.019220    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 11 00:08:10.071760 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.071728    2575 generic.go:356] "Generic (PLEG): container finished" podID="5832202a-b15c-4121-be80-6044399ed222" containerID="8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058" exitCode=0
Sep 11 00:08:10.071950 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.071795    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9"
Sep 11 00:08:10.071950 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.071806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" event={"ID":"5832202a-b15c-4121-be80-6044399ed222","Type":"ContainerDied","Data":"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"}
Sep 11 00:08:10.071950 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.071901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5g2q9" event={"ID":"5832202a-b15c-4121-be80-6044399ed222","Type":"ContainerDied","Data":"054ea17a1b02f9ff9865265bc84acddf8b0fd8b789d1a14cfc044e9f355cd47d"}
Sep 11 00:08:10.071950 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.071924    2575 scope.go:117] "RemoveContainer" containerID="8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"
Sep 11 00:08:10.079094 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.079066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") pod \"5832202a-b15c-4121-be80-6044399ed222\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") "
Sep 11 00:08:10.079229 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.079190    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert\") pod \"5832202a-b15c-4121-be80-6044399ed222\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") "
Sep 11 00:08:10.079354 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.079244    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb4z9\" (UniqueName: \"kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9\") pod \"5832202a-b15c-4121-be80-6044399ed222\" (UID: \"5832202a-b15c-4121-be80-6044399ed222\") "
Sep 11 00:08:10.081557 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.081523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5832202a-b15c-4121-be80-6044399ed222" (UID: "5832202a-b15c-4121-be80-6044399ed222"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:10.081682 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.081637    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9" (OuterVolumeSpecName: "kube-api-access-qb4z9") pod "5832202a-b15c-4121-be80-6044399ed222" (UID: "5832202a-b15c-4121-be80-6044399ed222"). InnerVolumeSpecName "kube-api-access-qb4z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:08:10.081745 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.081694    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5832202a-b15c-4121-be80-6044399ed222" (UID: "5832202a-b15c-4121-be80-6044399ed222"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:10.082822 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.082804    2575 scope.go:117] "RemoveContainer" containerID="8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"
Sep 11 00:08:10.083192 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:08:10.083172    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058\": container with ID starting with 8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058 not found: ID does not exist" containerID="8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"
Sep 11 00:08:10.083269 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.083201    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058"} err="failed to get container status \"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058\": rpc error: code = NotFound desc = could not find container \"8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058\": container with ID starting with 8c0795fa2dc45cd8bc0e0002a54a2079fa37ef054b2ff5c388a43a1b7d3a1058 not found: ID does not exist"
Sep 11 00:08:10.180785 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.180746    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.180785 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.180778    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5832202a-b15c-4121-be80-6044399ed222-webhook-cert\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.180785 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.180788    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb4z9\" (UniqueName: \"kubernetes.io/projected/5832202a-b15c-4121-be80-6044399ed222-kube-api-access-qb4z9\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.387693 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.387656    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 11 00:08:10.391676 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:10.391643    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5g2q9"]
Sep 11 00:08:12.228563 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:08:12.228453    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5832202a-b15c-4121-be80-6044399ed222" path="/var/lib/kubelet/pods/5832202a-b15c-4121-be80-6044399ed222/volumes"
Sep 11 00:09:11.154926 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.154877    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155546    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" containerName="maas-api"
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155571    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" containerName="maas-api"
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155584    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da110c99-f0a9-463c-8351-c2ec9f59ad15" containerName="maas-api"
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155592    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="da110c99-f0a9-463c-8351-c2ec9f59ad15" containerName="maas-api"
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155607    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" containerName="maas-api"
Sep 11 00:09:11.155615 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155615    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155638    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5832202a-b15c-4121-be80-6044399ed222" containerName="manager"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155645    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5832202a-b15c-4121-be80-6044399ed222" containerName="manager"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155671    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb7fc59-bf3b-4549-9e85-a3025455181c" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155679    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb7fc59-bf3b-4549-9e85-a3025455181c" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155690    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1164d7-52b0-420f-b114-7fe3a84afad3" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155698    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1164d7-52b0-420f-b114-7fe3a84afad3" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155813    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5832202a-b15c-4121-be80-6044399ed222" containerName="manager"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155836    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb7fc59-bf3b-4549-9e85-a3025455181c" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155848    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2d1537-3398-4152-b3ef-79e5bdb8aa9d" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155859    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0efc0ab-6b16-4db2-a4ee-fec3a78d2633" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155869    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="da110c99-f0a9-463c-8351-c2ec9f59ad15" containerName="maas-api"
Sep 11 00:09:11.155954 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.155881    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1164d7-52b0-420f-b114-7fe3a84afad3" containerName="maas-api"
Sep 11 00:09:11.159214 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.159190    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.162455 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.162428    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 00:09:11.162617 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.162529    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8t9fk\""
Sep 11 00:09:11.162617 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.162556    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 00:09:11.162617 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.162578    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 00:09:11.168124 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.167553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:11.196924 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.196883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.197148 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.196945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.197148 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.197099    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdknj\" (UniqueName: \"kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.298674 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.298626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdknj\" (UniqueName: \"kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.298902 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.298703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.298902 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.298762    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.301612 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.301578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.301748 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.301618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.306861 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.306837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdknj\" (UniqueName: \"kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj\") pod \"maas-controller-6cf5467c5b-5vwvk\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") " pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.472328 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.472230    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:11.752184 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.752158    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:11.754541 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:09:11.754460    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfee8f4_0686_4c48_a8bb_d29695db1f71.slice/crio-ab8e332270c833cedf0af16833a066a2caece535573a7f657aee21782f709987 WatchSource:0}: Error finding container ab8e332270c833cedf0af16833a066a2caece535573a7f657aee21782f709987: Status 404 returned error can't find the container with id ab8e332270c833cedf0af16833a066a2caece535573a7f657aee21782f709987
Sep 11 00:09:11.755884 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:11.755865    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 00:09:12.394257 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:12.394220    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" event={"ID":"8cfee8f4-0686-4c48-a8bb-d29695db1f71","Type":"ContainerStarted","Data":"ab8e332270c833cedf0af16833a066a2caece535573a7f657aee21782f709987"}
Sep 11 00:09:13.401339 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:13.401300    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" event={"ID":"8cfee8f4-0686-4c48-a8bb-d29695db1f71","Type":"ContainerStarted","Data":"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"}
Sep 11 00:09:13.401745 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:13.401375    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:13.419879 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:13.419823    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" podStartSLOduration=1.7175184300000002 podStartE2EDuration="2.419796918s" totalImagesPullingTime="702.278488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:09:13.416852973 +0000 UTC m=+1943.682849871" watchObservedRunningTime="2026-09-11 00:09:13.419796918 +0000 UTC m=+1943.685793788"
Sep 11 00:09:24.409452 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:24.409419    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:35.812801 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:35.812751    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:35.813506 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:35.813027    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" podUID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" containerName="manager" containerID="cri-o://42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81" gracePeriod=30
Sep 11 00:09:36.065843 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.065774    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:36.147611 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.147577    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs\") pod \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") "
Sep 11 00:09:36.147783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.147625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdknj\" (UniqueName: \"kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj\") pod \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") "
Sep 11 00:09:36.147783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.147683    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert\") pod \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\" (UID: \"8cfee8f4-0686-4c48-a8bb-d29695db1f71\") "
Sep 11 00:09:36.149919 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.149884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8cfee8f4-0686-4c48-a8bb-d29695db1f71" (UID: "8cfee8f4-0686-4c48-a8bb-d29695db1f71"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:09:36.150044 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.149931    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8cfee8f4-0686-4c48-a8bb-d29695db1f71" (UID: "8cfee8f4-0686-4c48-a8bb-d29695db1f71"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:09:36.150044 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.149956    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj" (OuterVolumeSpecName: "kube-api-access-hdknj") pod "8cfee8f4-0686-4c48-a8bb-d29695db1f71" (UID: "8cfee8f4-0686-4c48-a8bb-d29695db1f71"). InnerVolumeSpecName "kube-api-access-hdknj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:09:36.249046 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.249010    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-webhook-cert\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:09:36.249046 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.249041    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cfee8f4-0686-4c48-a8bb-d29695db1f71-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:09:36.249046 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.249051    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdknj\" (UniqueName: \"kubernetes.io/projected/8cfee8f4-0686-4c48-a8bb-d29695db1f71-kube-api-access-hdknj\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:09:36.526783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.526687    2575 generic.go:356] "Generic (PLEG): container finished" podID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" containerID="42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81" exitCode=0
Sep 11 00:09:36.526783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.526737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" event={"ID":"8cfee8f4-0686-4c48-a8bb-d29695db1f71","Type":"ContainerDied","Data":"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"}
Sep 11 00:09:36.526783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.526748    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk"
Sep 11 00:09:36.526783 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.526779    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-5vwvk" event={"ID":"8cfee8f4-0686-4c48-a8bb-d29695db1f71","Type":"ContainerDied","Data":"ab8e332270c833cedf0af16833a066a2caece535573a7f657aee21782f709987"}
Sep 11 00:09:36.527137 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.526801    2575 scope.go:117] "RemoveContainer" containerID="42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"
Sep 11 00:09:36.536676 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.536651    2575 scope.go:117] "RemoveContainer" containerID="42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"
Sep 11 00:09:36.537014 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:09:36.536990    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81\": container with ID starting with 42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81 not found: ID does not exist" containerID="42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"
Sep 11 00:09:36.537074 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.537025    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81"} err="failed to get container status \"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81\": rpc error: code = NotFound desc = could not find container \"42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81\": container with ID starting with 42d56ada874984ca97836f8d02d5b4990525779c233e96acc1bd6534f0a04c81 not found: ID does not exist"
Sep 11 00:09:36.542850 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.542818    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:36.547698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:36.547670    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-5vwvk"]
Sep 11 00:09:37.298842 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.298794    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-lsxzv"]
Sep 11 00:09:37.299270 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.299256    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" containerName="manager"
Sep 11 00:09:37.299320 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.299271    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" containerName="manager"
Sep 11 00:09:37.299360 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.299336    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" containerName="manager"
Sep 11 00:09:37.303895 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.303871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.306879 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.306855    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 00:09:37.307055 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.306855    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8t9fk\""
Sep 11 00:09:37.307055 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.306855    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 00:09:37.307055 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.306925    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 00:09:37.310358 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.310319    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-lsxzv"]
Sep 11 00:09:37.360020 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.359985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-webhook-cert\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.360238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.360045    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-metrics-certs\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.360238 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.360109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88nm\" (UniqueName: \"kubernetes.io/projected/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-kube-api-access-h88nm\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.460917 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.460877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88nm\" (UniqueName: \"kubernetes.io/projected/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-kube-api-access-h88nm\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.461135 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.460968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-webhook-cert\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.461135 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.461006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-metrics-certs\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.463733 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.463707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-webhook-cert\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.463907 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.463840    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-metrics-certs\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.469593 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.469558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88nm\" (UniqueName: \"kubernetes.io/projected/f0b8a2da-2d16-499b-8f4a-cdfa378b7e63-kube-api-access-h88nm\") pod \"maas-controller-6cf5467c5b-lsxzv\" (UID: \"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63\") " pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.617239 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.617148    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:37.911763 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:37.911686    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5467c5b-lsxzv"]
Sep 11 00:09:37.915663 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:09:37.915637    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b8a2da_2d16_499b_8f4a_cdfa378b7e63.slice/crio-6f17405d93a6a6a5e2d30bc8738f0be5f96f8b0a6d4553645107ab974d805a88 WatchSource:0}: Error finding container 6f17405d93a6a6a5e2d30bc8738f0be5f96f8b0a6d4553645107ab974d805a88: Status 404 returned error can't find the container with id 6f17405d93a6a6a5e2d30bc8738f0be5f96f8b0a6d4553645107ab974d805a88
Sep 11 00:09:38.228587 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:38.228480    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cfee8f4-0686-4c48-a8bb-d29695db1f71" path="/var/lib/kubelet/pods/8cfee8f4-0686-4c48-a8bb-d29695db1f71/volumes"
Sep 11 00:09:38.541066 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:38.541025    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv" event={"ID":"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63","Type":"ContainerStarted","Data":"25ea9adcda3ed873a9983a79ba70d697f06e64273407e3c89bb0aa1e3739c541"}
Sep 11 00:09:38.541616 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:38.541083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv" event={"ID":"f0b8a2da-2d16-499b-8f4a-cdfa378b7e63","Type":"ContainerStarted","Data":"6f17405d93a6a6a5e2d30bc8738f0be5f96f8b0a6d4553645107ab974d805a88"}
Sep 11 00:09:38.541616 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:38.541106    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:09:38.560703 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:38.560619    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv" podStartSLOduration=1.230285588 podStartE2EDuration="1.560593934s" totalImagesPullingTime="330.308346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:09:38.558266775 +0000 UTC m=+1968.824263661" watchObservedRunningTime="2026-09-11 00:09:38.560593934 +0000 UTC m=+1968.826590806"
Sep 11 00:09:49.548789 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:09:49.548695    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5467c5b-lsxzv"
Sep 11 00:12:46.614083 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.614048    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:12:46.618512 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.618463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.620835 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.620814    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f44b3dab-kserve-self-signed-certs\""
Sep 11 00:12:46.621813 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.621783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.621924 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.621842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.621924 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.621872    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.622003 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.621927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.622070 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.622046    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kckhc\" (UniqueName: \"kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.622126 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.622091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.631754 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.631727    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:12:46.722859 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.722818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723053 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.722896    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723053 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.722924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723053 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.722982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723053 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.723045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kckhc\" (UniqueName: \"kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723213 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.723080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723318 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.723298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723363 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.723334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.723401 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.723362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.725510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.725472    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.725706 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.725688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.731452 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.731429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kckhc\" (UniqueName: \"kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc\") pod \"e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") " pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:46.928671 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:46.928577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:47.200426 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:47.200397    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:12:47.202419 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:12:47.202391    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e1d2523_6846_4a58_a3ea_9473ad00ca99.slice/crio-d97d93a3a3f1ad78f63497e6eca84d7f02418c4cac3cf325469164bc00febb5e WatchSource:0}: Error finding container d97d93a3a3f1ad78f63497e6eca84d7f02418c4cac3cf325469164bc00febb5e: Status 404 returned error can't find the container with id d97d93a3a3f1ad78f63497e6eca84d7f02418c4cac3cf325469164bc00febb5e
Sep 11 00:12:47.515054 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:47.514964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" event={"ID":"7e1d2523-6846-4a58-a3ea-9473ad00ca99","Type":"ContainerStarted","Data":"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"}
Sep 11 00:12:47.515054 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:47.515017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" event={"ID":"7e1d2523-6846-4a58-a3ea-9473ad00ca99","Type":"ContainerStarted","Data":"d97d93a3a3f1ad78f63497e6eca84d7f02418c4cac3cf325469164bc00febb5e"}
Sep 11 00:12:47.532875 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:47.532823    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" podStartSLOduration=1.532807464 podStartE2EDuration="1.532807464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:12:47.530729804 +0000 UTC m=+2157.796726696" watchObservedRunningTime="2026-09-11 00:12:47.532807464 +0000 UTC m=+2157.798804335"
Sep 11 00:12:56.929293 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:56.929234    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:56.929293 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:56.929307    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:56.942041 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:56.942008    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:12:57.577226 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:12:57.577192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:13:02.310091 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:02.310051    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:13:17.331487 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.331439    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerName="main" containerID="cri-o://2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d" gracePeriod=45
Sep 11 00:13:17.586130 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.586060    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:13:17.677922 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.677881    2575 generic.go:356] "Generic (PLEG): container finished" podID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerID="2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d" exitCode=0
Sep 11 00:13:17.677922 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.677925    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"
Sep 11 00:13:17.678145 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.677930    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" event={"ID":"7e1d2523-6846-4a58-a3ea-9473ad00ca99","Type":"ContainerDied","Data":"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"}
Sep 11 00:13:17.678145 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.677972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" event={"ID":"7e1d2523-6846-4a58-a3ea-9473ad00ca99","Type":"ContainerDied","Data":"d97d93a3a3f1ad78f63497e6eca84d7f02418c4cac3cf325469164bc00febb5e"}
Sep 11 00:13:17.678145 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.677994    2575 scope.go:117] "RemoveContainer" containerID="2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"
Sep 11 00:13:17.686909 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.686890    2575 scope.go:117] "RemoveContainer" containerID="2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"
Sep 11 00:13:17.687227 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:13:17.687207    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d\": container with ID starting with 2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d not found: ID does not exist" containerID="2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"
Sep 11 00:13:17.687288 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.687240    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d"} err="failed to get container status \"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d\": rpc error: code = NotFound desc = could not find container \"2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d\": container with ID starting with 2faaf1687a7125ded1a6b602f8771bbdcb64bedfdd726a0d7df82cec2e52128d not found: ID does not exist"
Sep 11 00:13:17.725610 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725575    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kckhc\" (UniqueName: \"kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725780 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725620    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725780 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725667    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725780 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725743    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725945 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725945 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725874    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs\") pod \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\" (UID: \"7e1d2523-6846-4a58-a3ea-9473ad00ca99\") "
Sep 11 00:13:17.725945 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725927    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home" (OuterVolumeSpecName: "home") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:13:17.726218 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.725974    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:13:17.726218 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.726115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:13:17.726306 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.726263    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tmp-dir\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.726306 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.726284    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-model-cache\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.726306 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.726297    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-home\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.728100 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.728076    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:13:17.728228 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.728110    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm" (OuterVolumeSpecName: "dshm") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:13:17.728228 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.728114    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc" (OuterVolumeSpecName: "kube-api-access-kckhc") pod "7e1d2523-6846-4a58-a3ea-9473ad00ca99" (UID: "7e1d2523-6846-4a58-a3ea-9473ad00ca99"). InnerVolumeSpecName "kube-api-access-kckhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:13:17.826901 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.826860    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kckhc\" (UniqueName: \"kubernetes.io/projected/7e1d2523-6846-4a58-a3ea-9473ad00ca99-kube-api-access-kckhc\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.826901 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.826894    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1d2523-6846-4a58-a3ea-9473ad00ca99-dshm\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.826901 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.826903    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1d2523-6846-4a58-a3ea-9473ad00ca99-tls-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:17.998562 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:17.998522    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:13:18.000410 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:18.000390    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v"]
Sep 11 00:13:18.228108 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:18.228076    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" path="/var/lib/kubelet/pods/7e1d2523-6846-4a58-a3ea-9473ad00ca99/volumes"
Sep 11 00:13:21.520360 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.520318    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"]
Sep 11 00:13:21.521044 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.521024    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerName="main"
Sep 11 00:13:21.521044 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.521046    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerName="main"
Sep 11 00:13:21.521185 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.521115    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerName="main"
Sep 11 00:13:21.525670 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.525652    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.528165 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.528144    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h7jzs\""
Sep 11 00:13:21.532542 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.532517    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"]
Sep 11 00:13:21.668318 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.668279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtxj4\" (UniqueName: \"kubernetes.io/projected/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-kube-api-access-qtxj4\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.668537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.668340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-maas-api-tls\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.668537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.668421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-metrics-certs\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.668537 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.668456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-openshift-service-ca\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.769425 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.769393    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtxj4\" (UniqueName: \"kubernetes.io/projected/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-kube-api-access-qtxj4\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.769622 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.769447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-maas-api-tls\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.769622 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.769476    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-metrics-certs\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.769622 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.769519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-openshift-service-ca\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.770114 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.770080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-openshift-service-ca\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.772375 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.772297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-maas-api-tls\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.772375 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.772297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-metrics-certs\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.778123 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.778097    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtxj4\" (UniqueName: \"kubernetes.io/projected/b309917a-5177-4b25-8bd7-4bcf7af3fc9a-kube-api-access-qtxj4\") pod \"maas-api-6664b7b778-75p4t\" (UID: \"b309917a-5177-4b25-8bd7-4bcf7af3fc9a\") " pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:21.839254 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:21.839213    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:22.113888 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.113857    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"]
Sep 11 00:13:22.115539 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:13:22.115481    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb309917a_5177_4b25_8bd7_4bcf7af3fc9a.slice/crio-9568b66104acbe65073330d3e3b290ba272593505026ab6c10d31f0e8e406ea2 WatchSource:0}: Error finding container 9568b66104acbe65073330d3e3b290ba272593505026ab6c10d31f0e8e406ea2: Status 404 returned error can't find the container with id 9568b66104acbe65073330d3e3b290ba272593505026ab6c10d31f0e8e406ea2
Sep 11 00:13:22.567785 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.567750    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-f44b3dab-kserve-6bfcb969cf-4988v" podUID="7e1d2523-6846-4a58-a3ea-9473ad00ca99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Sep 11 00:13:22.705557 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.705519    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t" event={"ID":"b309917a-5177-4b25-8bd7-4bcf7af3fc9a","Type":"ContainerStarted","Data":"bdf39593d223d61cf561b45cce2adde503b200993e7b53c3a7afe8bc6589ebc1"}
Sep 11 00:13:22.705764 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.705575    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t" event={"ID":"b309917a-5177-4b25-8bd7-4bcf7af3fc9a","Type":"ContainerStarted","Data":"9568b66104acbe65073330d3e3b290ba272593505026ab6c10d31f0e8e406ea2"}
Sep 11 00:13:22.705764 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.705604    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:22.727744 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:22.727696    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t" podStartSLOduration=1.300653389 podStartE2EDuration="1.727676918s" totalImagesPullingTime="427.023529ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:13:22.726798777 +0000 UTC m=+2192.992795701" watchObservedRunningTime="2026-09-11 00:13:22.727676918 +0000 UTC m=+2192.993673790"
Sep 11 00:13:28.716109 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:28.716077    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6664b7b778-75p4t"
Sep 11 00:13:36.352510 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:36.352460    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 11 00:13:36.353156 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:36.352691    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" podUID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" containerName="maas-api" containerID="cri-o://300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb" gracePeriod=30
Sep 11 00:13:37.616710 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.616668    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 11 00:13:37.718745 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.718657    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgp6c\" (UniqueName: \"kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c\") pod \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") "
Sep 11 00:13:37.718745 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.718710    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") pod \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") "
Sep 11 00:13:37.718963 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.718817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") pod \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") "
Sep 11 00:13:37.718963 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.718911    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca\") pod \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\" (UID: \"37282bc7-3e5b-4cd0-b358-b3970f7a84e2\") "
Sep 11 00:13:37.719313 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.719288    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "37282bc7-3e5b-4cd0-b358-b3970f7a84e2" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:13:37.720986 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.720958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c" (OuterVolumeSpecName: "kube-api-access-hgp6c") pod "37282bc7-3e5b-4cd0-b358-b3970f7a84e2" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2"). InnerVolumeSpecName "kube-api-access-hgp6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:13:37.721632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.721613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "37282bc7-3e5b-4cd0-b358-b3970f7a84e2" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:13:37.721693 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.721642    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "37282bc7-3e5b-4cd0-b358-b3970f7a84e2" (UID: "37282bc7-3e5b-4cd0-b358-b3970f7a84e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:13:37.785359 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.785324    2575 generic.go:356] "Generic (PLEG): container finished" podID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" containerID="300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb" exitCode=0
Sep 11 00:13:37.785587 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.785368    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"
Sep 11 00:13:37.785587 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.785406    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" event={"ID":"37282bc7-3e5b-4cd0-b358-b3970f7a84e2","Type":"ContainerDied","Data":"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"}
Sep 11 00:13:37.785587 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.785459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm" event={"ID":"37282bc7-3e5b-4cd0-b358-b3970f7a84e2","Type":"ContainerDied","Data":"dc25a959dce462247db7db1bb73ae2b98a50ce18f6fd47c45f694934aab11ac7"}
Sep 11 00:13:37.785587 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.785523    2575 scope.go:117] "RemoveContainer" containerID="300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"
Sep 11 00:13:37.795297 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.795283    2575 scope.go:117] "RemoveContainer" containerID="300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"
Sep 11 00:13:37.795587 ip-10-0-134-115 kubenswrapper[2575]: E0911 00:13:37.795570    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb\": container with ID starting with 300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb not found: ID does not exist" containerID="300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"
Sep 11 00:13:37.795632 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.795599    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb"} err="failed to get container status \"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb\": rpc error: code = NotFound desc = could not find container \"300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb\": container with ID starting with 300a37056c4bd2c10d0f7e46a6ec1896f9ac5e00cdcebed136373a9fa7573edb not found: ID does not exist"
Sep 11 00:13:37.806575 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.806548    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 11 00:13:37.812575 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.812550    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-78cdb675f8-44ktm"]
Sep 11 00:13:37.820718 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.820698    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-metrics-certs\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:37.820812 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.820721    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-openshift-service-ca\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:37.820812 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.820730    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgp6c\" (UniqueName: \"kubernetes.io/projected/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-kube-api-access-hgp6c\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:37.820812 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:37.820741    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37282bc7-3e5b-4cd0-b358-b3970f7a84e2-maas-api-tls\") on node \"ip-10-0-134-115.ec2.internal\" DevicePath \"\""
Sep 11 00:13:38.228692 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:13:38.228656    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" path="/var/lib/kubelet/pods/37282bc7-3e5b-4cd0-b358-b3970f7a84e2/volumes"
Sep 11 00:14:01.254376 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:01.254321    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf5467c5b-lsxzv_f0b8a2da-2d16-499b-8f4a-cdfa378b7e63/manager/0.log"
Sep 11 00:14:01.616295 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:01.616262    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f676b6c5d-9sgns_f0025c96-154c-491b-ac9d-e2bf68fca28d/copy-manifests/0.log"
Sep 11 00:14:01.622942 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:01.622916    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f676b6c5d-9sgns_f0025c96-154c-491b-ac9d-e2bf68fca28d/manager/0.log"
Sep 11 00:14:02.299705 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:02.299677    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6664b7b778-75p4t_b309917a-5177-4b25-8bd7-4bcf7af3fc9a/maas-api/0.log"
Sep 11 00:14:02.517517 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:02.517467    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fmz25_292d3884-0456-450e-b188-93fcc10f2be7/postgres/0.log"
Sep 11 00:14:03.499744 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:03.499717    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mxjkt_dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1/manager/0.log"
Sep 11 00:14:03.629091 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:03.629062    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-kczh4_c241d1ed-366e-41a7-ba86-57d34ce1847d/kuadrant-console-plugin/0.log"
Sep 11 00:14:04.668523 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:04.668460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44b74856-nxglv_420ea471-84d8-42c0-9ba1-c60ad2564afb/kube-auth-proxy/0.log"
Sep 11 00:14:05.020290 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:05.020205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:14:05.133338 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:05.133305    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55546df84-vmltb_8c541dc9-c33b-483a-8502-7dcdff60c7fe/router/0.log"
Sep 11 00:14:05.589543 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:05.589485    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-h6bgb_53558d1b-f3e7-4cb5-b9f0-62182e454e60/main/0.log"
Sep 11 00:14:06.129283 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:06.129256    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-ltcd6_c758df08-adc3-4380-abcc-f9e5cdb64997/main/0.log"
Sep 11 00:14:15.115661 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:15.115589    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2jmnx_4a27b058-3fdc-4b2a-9d93-467eb73078a9/global-pull-secret-syncer/0.log"
Sep 11 00:14:15.307385 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:15.307353    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rq7pl_1f667796-980e-4052-8a46-1731c04e6c7c/konnectivity-agent/0.log"
Sep 11 00:14:15.326318 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:15.326289    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-115.ec2.internal_7a2bdc78eba34fc85c5ba40ae7a16c61/haproxy/0.log"
Sep 11 00:14:19.854746 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:19.854716    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mxjkt_dbc32c78-6c4b-4e26-ad7a-7bcfe4bd6de1/manager/0.log"
Sep 11 00:14:19.875889 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:19.875862    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-kczh4_c241d1ed-366e-41a7-ba86-57d34ce1847d/kuadrant-console-plugin/0.log"
Sep 11 00:14:21.583754 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.583722    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/alertmanager/0.log"
Sep 11 00:14:21.607475 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.607409    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/config-reloader/0.log"
Sep 11 00:14:21.625800 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.625779    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/kube-rbac-proxy-web/0.log"
Sep 11 00:14:21.644328 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.644307    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/kube-rbac-proxy/0.log"
Sep 11 00:14:21.662445 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.662426    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/kube-rbac-proxy-metric/0.log"
Sep 11 00:14:21.681665 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.681639    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/prom-label-proxy/0.log"
Sep 11 00:14:21.698932 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.698906    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_391756d2-42bb-4580-b22a-eba6b48132f2/init-config-reloader/0.log"
Sep 11 00:14:21.765113 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.765088    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-pv7d9_6c1df7c8-4c23-45d9-b873-865cce8c4bfa/kube-state-metrics/0.log"
Sep 11 00:14:21.782659 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.782628    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-pv7d9_6c1df7c8-4c23-45d9-b873-865cce8c4bfa/kube-rbac-proxy-main/0.log"
Sep 11 00:14:21.800236 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.800214    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-pv7d9_6c1df7c8-4c23-45d9-b873-865cce8c4bfa/kube-rbac-proxy-self/0.log"
Sep 11 00:14:21.825445 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.825422    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-775fb948db-9mzpc_4b6d0514-a6cb-4515-b240-32ee0281e346/metrics-server/0.log"
Sep 11 00:14:21.852798 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.852778    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-f47t8_4be27036-407a-4408-9f7d-a062184b68d2/monitoring-plugin/0.log"
Sep 11 00:14:21.886506 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.886430    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-264xv_a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb/node-exporter/0.log"
Sep 11 00:14:21.909207 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.909192    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-264xv_a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb/kube-rbac-proxy/0.log"
Sep 11 00:14:21.928612 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:21.928593    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-264xv_a456052d-0d2a-4fe0-91cb-ae6ac0c68dfb/init-textfile/0.log"
Sep 11 00:14:22.202119 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.202022    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/prometheus/0.log"
Sep 11 00:14:22.227440 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.227415    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/config-reloader/0.log"
Sep 11 00:14:22.247640 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.247619    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/thanos-sidecar/0.log"
Sep 11 00:14:22.266064 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.266042    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/kube-rbac-proxy-web/0.log"
Sep 11 00:14:22.286971 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.286939    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/kube-rbac-proxy/0.log"
Sep 11 00:14:22.305316 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.305290    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/kube-rbac-proxy-thanos/0.log"
Sep 11 00:14:22.325102 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.325082    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee1f0dea-72aa-43a3-bbd7-08cb9ac2df82/init-config-reloader/0.log"
Sep 11 00:14:22.367164 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.367131    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kffdk_89b006c8-0e17-4735-aa0c-df035c5224ae/prometheus-operator/0.log"
Sep 11 00:14:22.385351 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.385309    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kffdk_89b006c8-0e17-4735-aa0c-df035c5224ae/kube-rbac-proxy/0.log"
Sep 11 00:14:22.409791 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.409756    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-j54lv_fefb9e1b-a0d9-4ed8-a964-83f1587276fd/prometheus-operator-admission-webhook/0.log"
Sep 11 00:14:22.437422 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.437401    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9755dd4bf-7hp5z_22a78ef0-5b57-419a-998d-e19f47be5e71/telemeter-client/0.log"
Sep 11 00:14:22.456728 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.456646    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9755dd4bf-7hp5z_22a78ef0-5b57-419a-998d-e19f47be5e71/reload/0.log"
Sep 11 00:14:22.474357 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.474341    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9755dd4bf-7hp5z_22a78ef0-5b57-419a-998d-e19f47be5e71/kube-rbac-proxy/0.log"
Sep 11 00:14:22.504621 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.504587    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/thanos-query/0.log"
Sep 11 00:14:22.526731 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.526715    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/kube-rbac-proxy-web/0.log"
Sep 11 00:14:22.549631 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.549591    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/kube-rbac-proxy/0.log"
Sep 11 00:14:22.570132 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.570098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/prom-label-proxy/0.log"
Sep 11 00:14:22.601893 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.601857    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/kube-rbac-proxy-rules/0.log"
Sep 11 00:14:22.656772 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:22.656723    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5564ccdb75-lnc69_a1ecded2-81d8-436a-9a7a-0539697ada8b/kube-rbac-proxy-metrics/0.log"
Sep 11 00:14:25.409591 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.409561    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-8clcw_c666dc5c-ed83-4620-a4a9-954acdf30026/download-server/0.log"
Sep 11 00:14:25.656884 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.656852    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"]
Sep 11 00:14:25.657264 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.657252    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" containerName="maas-api"
Sep 11 00:14:25.657264 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.657265    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" containerName="maas-api"
Sep 11 00:14:25.657353 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.657344    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="37282bc7-3e5b-4cd0-b358-b3970f7a84e2" containerName="maas-api"
Sep 11 00:14:25.660562 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.660507    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.662588 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.662569    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-szhzc\"/\"default-dockercfg-fm6hk\""
Sep 11 00:14:25.662709 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.662690    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szhzc\"/\"kube-root-ca.crt\""
Sep 11 00:14:25.663333 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.663317    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szhzc\"/\"openshift-service-ca.crt\""
Sep 11 00:14:25.667830 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.667706    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"]
Sep 11 00:14:25.795384 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.795354    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94lr\" (UniqueName: \"kubernetes.io/projected/18ee1360-3ba4-41bd-ae61-ef93caf9c637-kube-api-access-x94lr\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.795581 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.795403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-sys\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.795581 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.795471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-proc\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.795581 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.795548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-podres\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.795698 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.795597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-lib-modules\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896142 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-proc\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896321 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-podres\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896321 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-lib-modules\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896321 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94lr\" (UniqueName: \"kubernetes.io/projected/18ee1360-3ba4-41bd-ae61-ef93caf9c637-kube-api-access-x94lr\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896321 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896233    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-proc\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896321 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-sys\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-podres\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896342    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-sys\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.896566 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.896373    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18ee1360-3ba4-41bd-ae61-ef93caf9c637-lib-modules\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.904192 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.904169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94lr\" (UniqueName: \"kubernetes.io/projected/18ee1360-3ba4-41bd-ae61-ef93caf9c637-kube-api-access-x94lr\") pod \"perf-node-gather-daemonset-7pgjx\" (UID: \"18ee1360-3ba4-41bd-ae61-ef93caf9c637\") " pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:25.971860 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:25.971788    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:26.118945 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:26.118904    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-tb4bz_33bc7525-926c-411b-9d7f-f7bc2378d10a/volume-data-source-validator/0.log"
Sep 11 00:14:26.240462 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:26.240436    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"]
Sep 11 00:14:26.242520 ip-10-0-134-115 kubenswrapper[2575]: W0911 00:14:26.242474    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18ee1360_3ba4_41bd_ae61_ef93caf9c637.slice/crio-cd4da81afc8148e81bb80368602247fc4d5325cc14c8c42aaeb3f40c55f9cec2 WatchSource:0}: Error finding container cd4da81afc8148e81bb80368602247fc4d5325cc14c8c42aaeb3f40c55f9cec2: Status 404 returned error can't find the container with id cd4da81afc8148e81bb80368602247fc4d5325cc14c8c42aaeb3f40c55f9cec2
Sep 11 00:14:27.049967 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.049924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx" event={"ID":"18ee1360-3ba4-41bd-ae61-ef93caf9c637","Type":"ContainerStarted","Data":"e8111d1b5cc44e54eafc8fd472379094e7869dcce4f049661aa40d23e21cb9d3"}
Sep 11 00:14:27.050380 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.050020    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx" event={"ID":"18ee1360-3ba4-41bd-ae61-ef93caf9c637","Type":"ContainerStarted","Data":"cd4da81afc8148e81bb80368602247fc4d5325cc14c8c42aaeb3f40c55f9cec2"}
Sep 11 00:14:27.050380 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.050051    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:27.065114 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.065070    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx" podStartSLOduration=2.065057154 podStartE2EDuration="2.065057154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:14:27.063996735 +0000 UTC m=+2257.329993660" watchObservedRunningTime="2026-09-11 00:14:27.065057154 +0000 UTC m=+2257.331054024"
Sep 11 00:14:27.329605 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.329515    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4wp8_058cc32b-8fa4-47de-a597-88d349640ecb/dns/0.log"
Sep 11 00:14:27.387234 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.387207    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4wp8_058cc32b-8fa4-47de-a597-88d349640ecb/kube-rbac-proxy/0.log"
Sep 11 00:14:27.613332 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:27.613256    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcttb_f409c393-b03d-4b8a-9250-3ce43782868a/dns-node-resolver/0.log"
Sep 11 00:14:28.164713 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:28.164682    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29818080-s6gs7_931e2487-bf61-45d1-b887-a0732a343188/image-pruner/0.log"
Sep 11 00:14:28.227187 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:28.227163    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rwlz_2ef2ea39-f2e5-446d-9485-ae649a729cd3/node-ca/0.log"
Sep 11 00:14:29.636987 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:29.636956    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44b74856-nxglv_420ea471-84d8-42c0-9ba1-c60ad2564afb/kube-auth-proxy/0.log"
Sep 11 00:14:29.752432 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:29.752403    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:14:29.773157 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:29.773128    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55546df84-vmltb_8c541dc9-c33b-483a-8502-7dcdff60c7fe/router/0.log"
Sep 11 00:14:30.668963 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:30.668927    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf4f5_d53c74b5-5763-455e-8e99-a6446c62421b/serve-healthcheck-canary/0.log"
Sep 11 00:14:31.265003 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:31.264978    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rrxm2_59546425-503d-457a-b4c9-fd5c40740f81/kube-rbac-proxy/0.log"
Sep 11 00:14:31.283107 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:31.283086    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rrxm2_59546425-503d-457a-b4c9-fd5c40740f81/exporter/0.log"
Sep 11 00:14:31.300785 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:31.300765    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rrxm2_59546425-503d-457a-b4c9-fd5c40740f81/extractor/0.log"
Sep 11 00:14:33.065203 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:33.065175    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-szhzc/perf-node-gather-daemonset-7pgjx"
Sep 11 00:14:33.744611 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:33.744583    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf5467c5b-lsxzv_f0b8a2da-2d16-499b-8f4a-cdfa378b7e63/manager/0.log"
Sep 11 00:14:33.903881 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:33.903851    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f676b6c5d-9sgns_f0025c96-154c-491b-ac9d-e2bf68fca28d/manager/0.log"
Sep 11 00:14:33.925103 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:33.925082    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f676b6c5d-9sgns_f0025c96-154c-491b-ac9d-e2bf68fca28d/copy-manifests/0.log"
Sep 11 00:14:35.582438 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:35.582408    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f59994447-7s9v8_010a610b-54c4-4b9d-b3e9-09b74c9de517/manager/0.log"
Sep 11 00:14:37.182222 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:37.182136    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44b74856-nxglv_420ea471-84d8-42c0-9ba1-c60ad2564afb/kube-auth-proxy/0.log"
Sep 11 00:14:37.275072 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:37.275041    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:14:37.287217 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:37.287192    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55546df84-vmltb_8c541dc9-c33b-483a-8502-7dcdff60c7fe/router/0.log"
Sep 11 00:14:39.284653 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:39.284626    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44b74856-nxglv_420ea471-84d8-42c0-9ba1-c60ad2564afb/kube-auth-proxy/0.log"
Sep 11 00:14:39.380031 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:39.380003    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qv9kd_cc81f5fb-b916-4501-ab64-28f1aa60dc30/payload-processing/0.log"
Sep 11 00:14:39.400205 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:39.400168    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55546df84-vmltb_8c541dc9-c33b-483a-8502-7dcdff60c7fe/router/0.log"
Sep 11 00:14:40.974082 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:40.974038    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qhfjg_0455cf93-2914-4f8d-aae4-0d2c6494f4fe/migrator/0.log"
Sep 11 00:14:40.991623 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:40.991594    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qhfjg_0455cf93-2914-4f8d-aae4-0d2c6494f4fe/graceful-termination/0.log"
Sep 11 00:14:41.342921 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:41.342887    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-b8v84_a8802c81-0116-4bb9-abbe-f2da5c7cb762/kube-storage-version-migrator-operator/1.log"
Sep 11 00:14:41.343927 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:41.343904    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-b8v84_a8802c81-0116-4bb9-abbe-f2da5c7cb762/kube-storage-version-migrator-operator/0.log"
Sep 11 00:14:42.496690 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.496663    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/kube-multus-additional-cni-plugins/0.log"
Sep 11 00:14:42.517534 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.517504    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/egress-router-binary-copy/0.log"
Sep 11 00:14:42.538094 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.538067    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/cni-plugins/0.log"
Sep 11 00:14:42.556424 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.556399    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/bond-cni-plugin/0.log"
Sep 11 00:14:42.573337 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.573320    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/routeoverride-cni/0.log"
Sep 11 00:14:42.590769 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.590752    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/whereabouts-cni-bincopy/0.log"
Sep 11 00:14:42.607836 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.607820    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vvdfz_0a93e61a-186f-4b2b-ba76-0e1498e04f67/whereabouts-cni/0.log"
Sep 11 00:14:42.791542 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:42.791515    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gf86q_a9fec15c-141b-4faf-84f6-483fd55b9252/kube-multus/0.log"
Sep 11 00:14:43.003277 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:43.003231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gq8xl_a8648551-0047-47b8-a0af-291fdc3b8577/network-metrics-daemon/0.log"
Sep 11 00:14:43.022836 ip-10-0-134-115 kubenswrapper[2575]: I0911 00:14:43.022808    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gq8xl_a8648551-0047-47b8-a0af-291fdc3b8577/kube-rbac-proxy/0.log"