Sep 07 16:17:37.495672 ip-10-0-135-154 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 16:17:37.495685 ip-10-0-135-154 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 16:17:37.495694 ip-10-0-135-154 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 16:17:37.496084 ip-10-0-135-154 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 16:17:46.598642 ip-10-0-135-154 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 07 16:17:46.598656 ip-10-0-135-154 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 57608d9f7c794a12bb37dafff4b4b88e --
Sep 07 16:19:21.252027 ip-10-0-135-154 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 16:19:21.674127 ip-10-0-135-154 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 16:19:21.674127 ip-10-0-135-154 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 16:19:21.674127 ip-10-0-135-154 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 16:19:21.674127 ip-10-0-135-154 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 16:19:21.674127 ip-10-0-135-154 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 16:19:21.675053 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.674964    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 16:19:21.680779 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680764    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.680779 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680779    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.680779 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680782    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680786    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680789    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680791    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680794    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680797    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680801    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680803    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680806    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680809    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680811    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680814    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680817    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680819    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680822    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680825    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680827    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680830    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680832    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680835    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.680870 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680838    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680840    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680843    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680845    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680848    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680850    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680853    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680856    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680859    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680861    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680871    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680873    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680885    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680889    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680892    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680895    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680897    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680900    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680903    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680905    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.681356 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680908    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680910    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680913    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680917    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680919    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680922    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680924    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680928    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680930    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680933    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680936    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680939    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680941    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680944    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680947    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680950    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680953    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680955    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680959    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.681852 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680962    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680964    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680967    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680969    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680976    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680979    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680981    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680984    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680986    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680989    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680991    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680994    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680996    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.680998    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681001    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681004    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681006    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681009    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681014    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.682354 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681019    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681024    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681027    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681030    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681034    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681037    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681040    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681043    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681047    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681050    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681053    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681056    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681059    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681062    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681065    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681068    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681071    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681074    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681077    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681783    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.682850 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681791    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681794    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681797    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681801    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681803    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681806    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681809    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681812    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681815    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681817    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681820    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681822    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681825    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681827    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681830    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681833    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681836    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681838    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681840    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681843    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.683412 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681846    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681849    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681851    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681854    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681856    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681859    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681862    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681865    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681867    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681870    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681873    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681876    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681878    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681881    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681883    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681886    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681888    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681891    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681893    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681896    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.683945 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681898    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681901    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681903    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681906    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681910    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681915    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681918    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681921    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681923    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681926    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681928    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681931    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681934    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681937    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681939    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681942    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681945    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681947    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681952    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.684426 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681956    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681959    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681962    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681965    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681968    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681971    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681974    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681977    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681979    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681982    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681985    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681987    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681989    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681993    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681996    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.681998    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682000    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682003    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682005    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.684917 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682009    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682012    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682014    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682017    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682020    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682022    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682027    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682031    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682033    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682036    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682039    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682041    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682044    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682047    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682049    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682051    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682054    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682056    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682059    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682062    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.685377 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682436    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682441    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682443    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682446    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682449    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682451    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682454    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682457    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682461    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682464    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682467    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682469    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682472    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682475    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682478    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682480    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682483    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682486    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682489    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.685878 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682492    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682494    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682497    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682500    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682502    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682505    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682507    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682510    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682512    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682515    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682517    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682520    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682523    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682526    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682528    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682531    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682533    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682538    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682542    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.686342 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682545    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682548    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682550    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682553    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682555    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682558    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682561    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682564    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682566    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682569    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682572    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682574    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682577    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682579    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682582    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682584    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682587    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682589    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682592    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682594    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.686817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682597    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682600    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682602    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682604    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682607    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682610    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682613    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682616    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682618    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682621    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682623    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682627    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682630    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682633    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682636    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682638    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682643    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682646    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682649    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682652    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.687332 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682655    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682658    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682661    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682664    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682667    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682670    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682673    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682675    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682678    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682681    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682683    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682686    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682688    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682691    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682693    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682696    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682698    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682701    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682703    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682706    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.687864 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.682709    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682796    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682832    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682838    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682843    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682848    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682852    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682857    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682863    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682867    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682870    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682873    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682877    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682880    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682883    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682886    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682889    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682892    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682895    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682899    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682902    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682905    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682908    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682912    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 16:19:21.688356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682915    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682918    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682922    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682925    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682928    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682931    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682934    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682937    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682940    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682944    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682948    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682950    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682953    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682956    2579 flags.go:64] FLAG: --enable-server="true"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682959    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682963    2579 flags.go:64] FLAG: --event-burst="100"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682968    2579 flags.go:64] FLAG: --event-qps="50"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682971    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682974    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682977    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682981    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682984    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682987    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682990    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682996    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 16:19:21.688981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.682999    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683002    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683005    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683008    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683011    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683013    2579 flags.go:64] FLAG: --feature-gates=""
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683018    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683021    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683024    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683027    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683030    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683033    2579 flags.go:64] FLAG: --help="false"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683036    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-135-154.ec2.internal"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683039    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683043    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683045    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683050    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683053    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683056    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683059    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683062    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683064    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683067    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683072    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 16:19:21.689585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683074    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683077    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683080    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683083    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683086    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683089    2579 flags.go:64] FLAG: --lock-file=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683092    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683096    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683099    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683105    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683107    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683110    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683114    2579 flags.go:64] FLAG: --logging-format="text"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683116    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683120    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683123    2579 flags.go:64] FLAG: --manifest-url=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683126    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683130    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683133    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683136    2579 flags.go:64] FLAG: --max-pods="110"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683140    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683142    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683145    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683148    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683151    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 16:19:21.690185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683155    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683158    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683168    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683171    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683174    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683178    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683182    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683187    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683190    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683193    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683196    2579 flags.go:64] FLAG: --port="10250"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683199    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683202    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0658227059064a391"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683205    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683210    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683213    2579 flags.go:64] FLAG: --register-node="true"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683216    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683219    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683222    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683225    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683228    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683232    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683234    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683237    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683240    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 16:19:21.690851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683243    2579 flags.go:64] FLAG: --runonce="false"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683246    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683249    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683252    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683255    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683258    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683261    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683264    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683267    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683270    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683273    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683276    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683279    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683282    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683286    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683289    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683294    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683297    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683300    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683304    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683307    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683311    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683314    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683317    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683320    2579 flags.go:64] FLAG: --v="2"
Sep 07 16:19:21.691497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683324    2579 flags.go:64] FLAG: --version="false"
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683329    2579 flags.go:64] FLAG: --vmodule=""
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683332    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.683336    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683471    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683478    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683481    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683486    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683489    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683493    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683496    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683499    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683502    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683505    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683507    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683510    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683519    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683522    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683524    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683527    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.692397 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683530    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683533    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683535    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683538    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683540    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683543    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683546    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683548    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683552    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683555    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683557    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683560    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683562    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683565    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683567    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683570    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683572    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683575    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683578    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683580    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.693051 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683583    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683586    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683588    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683591    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683593    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683596    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683599    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683601    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683604    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683613    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683616    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683618    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683621    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683624    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683627    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683629    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683633    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683642    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683645    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.693523 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683648    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683651    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683654    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683657    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683659    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683662    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683665    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683667    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683670    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683673    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683675    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683678    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683680    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683683    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683686    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683688    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683692    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683696    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683699    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683702    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.693998 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683704    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683707    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683710    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683718    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683721    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683724    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683727    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683729    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683731    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683734    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683737    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683739    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683756    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683760    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683763    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683765    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683768    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683770    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683773    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683775    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.694498 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683778    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683781    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683784    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.683786    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.684469    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.693944    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.693959    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.693974    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.693980    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694041    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694046    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694049    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694052    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694055    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.695038 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694058    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694061    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694063    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694066    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694068    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694071    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694073    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694076    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694079    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694082    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694085    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694088    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694091    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694093    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694096    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694098    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694101    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694104    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694107    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694110    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.695385 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694112    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694115    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694118    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694121    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694124    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694126    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694129    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694132    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694134    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694137    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694139    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694142    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694144    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694147    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694149    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694152    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694154    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694157    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694159    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694162    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.695879 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694164    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694173    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694175    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694178    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694181    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694183    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694186    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694188    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694191    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694193    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694196    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694198    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694201    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694203    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694206    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694208    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694211    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694213    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694216    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694218    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.696362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694221    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694223    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694226    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694228    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694231    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694233    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694236    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694239    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694241    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694244    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694246    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694250    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694254    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694257    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694260    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694263    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694265    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694268    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694271    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694273    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.697002 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694276    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694279    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694281    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694284    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694286    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694290    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694294    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694298    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694301    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694304    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694307    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694310    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694313    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.697479 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694315    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.694320    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694441    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694447    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694450    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694453    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694456    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694459    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694462    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694464    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694467    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694470    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694473    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694476    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694481    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 16:19:21.697817 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694485    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694488    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694491    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694495    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694498    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694501    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694504    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694507    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694510    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694513    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694516    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694518    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694520    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694523    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694526    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694528    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694531    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694533    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694536    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 16:19:21.698192 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694538    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694541    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694543    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694546    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694548    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694551    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694553    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694556    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694558    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694561    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694563    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694566    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694568    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694572    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694575    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694577    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694580    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694582    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694585    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694587    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 16:19:21.698655 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694590    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694593    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694595    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694598    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694601    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694603    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694606    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694608    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694611    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694614    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694616    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694619    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694621    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694624    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694626    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694628    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694631    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694633    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694636    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694638    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 16:19:21.699146 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694641    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694643    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694646    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694648    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694653    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694656    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694659    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694662    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694664    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694667    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694670    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694673    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694676    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694678    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694681    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694683    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694686    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694689    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694691    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 16:19:21.699632 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694694    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694696    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694698    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694701    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694703    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694706    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694708    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:21.694710    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.694716    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.694862    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.698260    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.699333    2579 server.go:1019] "Starting client certificate rotation"
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.699439    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 16:19:21.700106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.699475    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 16:19:21.702295 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.702280    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 16:19:21.704390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.704373    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 16:19:21.711512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.711496    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 07 16:19:21.714808 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.714795    2579 log.go:25] "Validated CRI v1 image API"
Sep 07 16:19:21.715656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.715636    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 16:19:21.720824 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.720806    2579 fs.go:135] Filesystem UUIDs: map[43653833-7599-4c03-bbae-b13b782fa658:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b958005e-2826-4046-a461-e55c8ea5dc39:/dev/nvme0n1p4]
Sep 07 16:19:21.720901 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.720823    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 16:19:21.727758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.727618    2579 manager.go:216] Machine: {Timestamp:2026-09-07 16:19:21.725906294 +0000 UTC m=+0.370201596 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108763 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20479ade210cb4125f0132449ca68f SystemUUID:ec20479a-de21-0cb4-125f-0132449ca68f BootID:57608d9f-7c79-4a12-bb37-dafff4b4b88e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:e5:7d:77:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:e5:7d:77:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:7d:eb:f4:55:17 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 16:19:21.727758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.727728    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 16:19:21.727904 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.727867    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 16:19:21.729505 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.729483    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 16:19:21.731692 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.731668    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 16:19:21.731868 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.731698    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 16:19:21.732538 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.732526    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 16:19:21.732596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.732544    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 16:19:21.732596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.732555    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 16:19:21.733385 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.733373    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 16:19:21.735237 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.735224    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 16:19:21.739812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.739798    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 16:19:21.739944 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.739933    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 16:19:21.740010 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.739971    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 16:19:21.740010 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.739984    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 16:19:21.740010 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.739999    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 16:19:21.740130 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.740012    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 07 16:19:21.740130 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.740024    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 16:19:21.740395 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.740382    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 16:19:21.740455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.740405    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 16:19:21.744172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.744157    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 16:19:21.747187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747172    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 16:19:21.747249 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747204    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 16:19:21.747374 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747364    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 16:19:21.747417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747380    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 16:19:21.747417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747386    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 16:19:21.747417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747392    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 16:19:21.747417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747400    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 16:19:21.747417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.747408    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 16:19:21.748015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748006    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 16:19:21.748056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748016    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 16:19:21.748056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748023    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 16:19:21.748056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748029    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 16:19:21.748056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748037    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 16:19:21.748056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.748045    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 16:19:21.749828 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.749816    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 16:19:21.749828 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.749828    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 16:19:21.751667 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.751637    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-154.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 16:19:21.751772 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.751694    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 16:19:21.751772 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.751704    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 16:19:21.753738 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.753724    2579 server.go:1268] "Started kubelet"
Sep 07 16:19:21.753833 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.753807    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 16:19:21.753939 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.753897    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 16:19:21.753985 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.753959    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 16:19:21.755903 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.755880    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 16:19:21.756192 ip-10-0-135-154 systemd[1]: Started Kubernetes Kubelet.
Sep 07 16:19:21.756526 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.756214    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 16:19:21.761238 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.761211    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 16:19:21.762557 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.762535    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 16:19:21.762557 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.762541    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 16:19:21.762679 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.762625    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:21.763472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763309    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 16:19:21.763549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763352    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 16:19:21.763549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763491    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 16:19:21.763646 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763581    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 16:19:21.763646 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763591    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 16:19:21.763850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763835    2579 factory.go:153] Registering CRI-O factory
Sep 07 16:19:21.763920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763853    2579 factory.go:223] Registration of the crio container factory successfully
Sep 07 16:19:21.763920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763905    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 16:19:21.763920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763915    2579 factory.go:55] Registering systemd factory
Sep 07 16:19:21.764044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763922    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 07 16:19:21.764044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763939    2579 factory.go:103] Registering Raw factory
Sep 07 16:19:21.764044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.763953    2579 manager.go:1196] Started watching for new ooms in manager
Sep 07 16:19:21.764339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.764328    2579 manager.go:349] Starting recovery of all containers
Sep 07 16:19:21.767527 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.767481    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 16:19:21.767527 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.767514    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 16:19:21.768107 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.768062    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7s267"
Sep 07 16:19:21.773059 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.767409    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-154.ec2.internal.18d31644cd458b62  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-154.ec2.internal,UID:ip-10-0-135-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-154.ec2.internal,},FirstTimestamp:2026-09-07 16:19:21.753693026 +0000 UTC m=+0.397988327,LastTimestamp:2026-09-07 16:19:21.753693026 +0000 UTC m=+0.397988327,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-154.ec2.internal,}"
Sep 07 16:19:21.774693 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.774669    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7s267"
Sep 07 16:19:21.778187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.778164    2579 manager.go:354] Recovery completed
Sep 07 16:19:21.782354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.782342    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.784345 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784330    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.784411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784360    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.784411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784374    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.784922 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784908    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 16:19:21.784922 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784921    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 16:19:21.785034 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784936    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.785034 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.784941    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 16:19:21.785566 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.785551    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.785615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.785583    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.785615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.785593    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.786074 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.785998    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-154.ec2.internal.18d31644cf19480a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-154.ec2.internal,UID:ip-10-0-135-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-154.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-154.ec2.internal,},FirstTimestamp:2026-09-07 16:19:21.784346634 +0000 UTC m=+0.428641935,LastTimestamp:2026-09-07 16:19:21.784346634 +0000 UTC m=+0.428641935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-154.ec2.internal,}"
Sep 07 16:19:21.786966 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.786953    2579 policy_none.go:49] "None policy: Start"
Sep 07 16:19:21.787021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.786973    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 16:19:21.787021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.786988    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 16:19:21.788964 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.788951    2579 policy_none.go:47] "Start"
Sep 07 16:19:21.825374 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825357    2579 manager.go:337] "Starting Device Plugin manager"
Sep 07 16:19:21.825491 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.825394    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 16:19:21.825491 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825409    2579 server.go:87] "Starting device plugin registration server"
Sep 07 16:19:21.825785 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825773    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 16:19:21.825839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825786    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 16:19:21.825893 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825881    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 16:19:21.825976 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825961    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 16:19:21.826021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.825977    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 16:19:21.826533 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.826514    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 16:19:21.826606 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.826571    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:21.879873 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.879843    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 16:19:21.882104 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.881133    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 16:19:21.882104 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.881174    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 16:19:21.882104 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.881201    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 16:19:21.882104 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.881237    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 16:19:21.883456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.883432    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 16:19:21.926408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.926328    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.927253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.927238    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.927324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.927266    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.927324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.927276    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.927324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.927299    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-154.ec2.internal"
Sep 07 16:19:21.936026 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.936007    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-154.ec2.internal"
Sep 07 16:19:21.936093 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.936029    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-154.ec2.internal\": node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:21.980658 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:21.980630    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:21.981619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.981599    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"]
Sep 07 16:19:21.981674 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.981664    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.982584 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.982567    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.982671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.982598    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.982671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.982613    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.984961 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.984946    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.985125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:21.985173 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985139    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.985856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985840    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.985916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985844    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.985916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985893    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.985916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985908    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.986002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985871    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.986002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.985948    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:21.988215 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.988200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:21.988253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.988232    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 16:19:21.989018 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.989002    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 16:19:21.989119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.989023    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 16:19:21.989119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:21.989038    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeHasSufficientPID"
Sep 07 16:19:22.004351 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.004328    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-154.ec2.internal\" not found" node="ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.008710 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.008696    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-154.ec2.internal\" not found" node="ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.065471 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.065443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.065575 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.065489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.065575 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.065504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67dd56507bf34e3ef1ec1fea9b6f36e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"67dd56507bf34e3ef1ec1fea9b6f36e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.080727 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.080702    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.166369 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.166456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.166456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67dd56507bf34e3ef1ec1fea9b6f36e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"67dd56507bf34e3ef1ec1fea9b6f36e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.166456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.166544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89dcf88f0bd9b87ca70ae75da4c728ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal\" (UID: \"89dcf88f0bd9b87ca70ae75da4c728ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.166544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.166508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67dd56507bf34e3ef1ec1fea9b6f36e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-154.ec2.internal\" (UID: \"67dd56507bf34e3ef1ec1fea9b6f36e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.181495 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.181442    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.282325 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.282287    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.307461 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.307437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.311161 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.311142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:22.383275 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.383229    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.483887 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.483810    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.584436 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.584401    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.684944 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.684906    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.699101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.699060    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 16:19:22.699243 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.699227    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 16:19:22.740972 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.740906    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 16:19:22.778257 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.778220    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 16:14:21 +0000 UTC" deadline="2028-03-09 05:21:00.279627803 +0000 UTC"
Sep 07 16:19:22.778257 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.778253    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13165h1m37.501378094s"
Sep 07 16:19:22.785396 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.785363    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.886040 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.886007    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:22.917881 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:22.917836    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89dcf88f0bd9b87ca70ae75da4c728ad.slice/crio-3818237894d64fbae791be94f6557afbd479548a6899d5239b6d4d72c02c8fb9 WatchSource:0}: Error finding container 3818237894d64fbae791be94f6557afbd479548a6899d5239b6d4d72c02c8fb9: Status 404 returned error can't find the container with id 3818237894d64fbae791be94f6557afbd479548a6899d5239b6d4d72c02c8fb9
Sep 07 16:19:22.918141 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:22.918121    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67dd56507bf34e3ef1ec1fea9b6f36e9.slice/crio-29473bf4a4d4ff393effbc972b28ec7aad9e14326eeecdf55f0f1688bb67a84e WatchSource:0}: Error finding container 29473bf4a4d4ff393effbc972b28ec7aad9e14326eeecdf55f0f1688bb67a84e: Status 404 returned error can't find the container with id 29473bf4a4d4ff393effbc972b28ec7aad9e14326eeecdf55f0f1688bb67a84e
Sep 07 16:19:22.924609 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:22.924589    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:19:22.986993 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:22.986946    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-154.ec2.internal\" not found"
Sep 07 16:19:23.048935 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.048861    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 16:19:23.063536 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.063512    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:23.074888 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.074869    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 16:19:23.076499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.076487    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal"
Sep 07 16:19:23.081596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.081584    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 16:19:23.206091 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.206063    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 16:19:23.520396 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.520308    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 16:19:23.740553 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.740517    2579 apiserver.go:52] "Watching apiserver"
Sep 07 16:19:23.747058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.746853    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 16:19:23.747393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.747364    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-6hbb7","openshift-multus/multus-additional-cni-plugins-2zdjd","openshift-ovn-kubernetes/ovnkube-node-vsjqp","kube-system/konnectivity-agent-p5n2k","kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh","openshift-cluster-node-tuning-operator/tuned-whj92","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal","openshift-multus/network-metrics-daemon-zj7m6","openshift-network-diagnostics/network-check-target-pc54l","openshift-network-operator/iptables-alerter-w5hn5","kube-system/global-pull-secret-syncer-89lj5","openshift-dns/node-resolver-zhl2d","openshift-image-registry/node-ca-xcftb"]
Sep 07 16:19:23.750510 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.750487    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.753026 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.753002    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.753317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.753296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rt544\""
Sep 07 16:19:23.753550 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.753525    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.754789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.754767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.757163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.757140    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.757275 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.757198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 16:19:23.757412 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.757394    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-drkrr\""
Sep 07 16:19:23.757534 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.757516    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.757705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.757689    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.759773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.759754    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.759888 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.759792    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 16:19:23.759952 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.759937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.760109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.759758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h6rwh\""
Sep 07 16:19:23.760412 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.760298    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.761857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.760902    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 16:19:23.761857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.761167    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 16:19:23.761857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.761488    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 16:19:23.762497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.762136    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nvr2w\""
Sep 07 16:19:23.762497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.762231    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 16:19:23.762497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.762436    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 16:19:23.763025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.763007    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 16:19:23.763607 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.763583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.766209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.765788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.766209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.766090    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6z7w5\""
Sep 07 16:19:23.766209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.766102    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.766209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.766141    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 16:19:23.768228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.766677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.769101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.768746    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.769101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.768764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 16:19:23.769252 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.769147    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.769463 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.769444    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4k8xn\""
Sep 07 16:19:23.770207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.770187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:23.770306 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.770278    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:23.773270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.773219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.775522 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.775495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mkxmh\""
Sep 07 16:19:23.775790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.775752    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.775907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.775869    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.776093 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.776070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.777761 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.777872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-sys\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.777872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r742v\" (UniqueName: \"kubernetes.io/projected/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-kube-api-access-r742v\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.777872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-node-log\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnlwl\" (UniqueName: \"kubernetes.io/projected/6c064c9c-2559-47fd-a423-d39d6d5f6b25-kube-api-access-pnlwl\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-serviceca\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpj5f\" (UniqueName: \"kubernetes.io/projected/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-kube-api-access-dpj5f\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.777989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-var-lib-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778019    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-systemd\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-run\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-tuned\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp8mz\" (UniqueName: \"kubernetes.io/projected/8c5441c1-42df-46c1-a4db-f41190a0f2a8-kube-api-access-vp8mz\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-systemd-units\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778190    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:23.778339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-netd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.778336    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-modprobe-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-var-lib-kubelet\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c5441c1-42df-46c1-a4db-f41190a0f2a8-iptables-alerter-script\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c5441c1-42df-46c1-a4db-f41190a0f2a8-host-slash\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-config\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778537    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-conf\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-kubelet\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778628    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-sys-fs\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.778768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-log-socket\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-env-overrides\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-script-lib\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-device-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778860    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-etc-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysconfig\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778929    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-systemd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.778954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-bin\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779030    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 16:19:23.779225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovn-node-metrics-cert\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-agent-certs\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-konnectivity-ca\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779269    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-registration-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-lib-modules\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-host\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-tmp\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779449    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t6tcq\""
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-slash\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-ovn\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-host\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-socket-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmdmn\" (UniqueName: \"kubernetes.io/projected/54ddde38-4866-430b-a36a-ac60a8230e57-kube-api-access-kmdmn\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-kubernetes\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.779728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.779666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-netns\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.782094 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.782071    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 16:19:23.784020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.783998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.786106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.786044    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 16:19:23.786435 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.786417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.786517 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.786480    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:23.786722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.786704    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dkkpw\""
Sep 07 16:19:23.800760 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.800735    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nwsrr"
Sep 07 16:19:23.809537 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.808491    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nwsrr"
Sep 07 16:19:23.864479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.864447    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 16:19:23.880949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.880901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-node-log\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.880949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.880951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.880980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-serviceca\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpj5f\" (UniqueName: \"kubernetes.io/projected/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-kube-api-access-dpj5f\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-sys-fs\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cni-binary-copy\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-var-lib-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-run\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.881146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-tuned\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-systemd-units\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-config\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-multus-certs\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-etc-kubernetes\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-modprobe-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-k8s-cni-cncf-io\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-multus\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-dbus\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.881388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-system-cni-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-log-socket\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-conf-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-etc-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysconfig\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-systemd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-bin\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-konnectivity-ca\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-bin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-tmp\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-slash\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-ovn\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec32b97f-faca-4bcb-8733-9b95a727a079-hosts-file\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.881773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-kubelet-config\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-netns\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cnibin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-hostroot\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-sys\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r742v\" (UniqueName: \"kubernetes.io/projected/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-kube-api-access-r742v\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnlwl\" (UniqueName: \"kubernetes.io/projected/6c064c9c-2559-47fd-a423-d39d6d5f6b25-kube-api-access-pnlwl\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.881988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-cnibin\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-systemd\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp8mz\" (UniqueName: \"kubernetes.io/projected/8c5441c1-42df-46c1-a4db-f41190a0f2a8-kube-api-access-vp8mz\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-netd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6gl\" (UniqueName: \"kubernetes.io/projected/d81a54d9-b6b8-4822-b998-e823d5ba34f0-kube-api-access-jx6gl\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-netns\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.882371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-var-lib-kubelet\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c5441c1-42df-46c1-a4db-f41190a0f2a8-iptables-alerter-script\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c5441c1-42df-46c1-a4db-f41190a0f2a8-host-slash\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-socket-dir-parent\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-conf\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-kubelet\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-system-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-daemon-config\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-env-overrides\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-script-lib\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-device-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7lhp\" (UniqueName: \"kubernetes.io/projected/1edc37cf-dc0d-4e09-977f-30a393e7be84-kube-api-access-m7lhp\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovn-node-metrics-cert\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-agent-certs\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-registration-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmdmn\" (UniqueName: \"kubernetes.io/projected/54ddde38-4866-430b-a36a-ac60a8230e57-kube-api-access-kmdmn\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec32b97f-faca-4bcb-8733-9b95a727a079-tmp-dir\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfkg9\" (UniqueName: \"kubernetes.io/projected/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-kube-api-access-tfkg9\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-lib-modules\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-host\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-host\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.882991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-socket-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-kubelet\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-kubernetes\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvqpg\" (UniqueName: \"kubernetes.io/projected/ec32b97f-faca-4bcb-8733-9b95a727a079-kube-api-access-bvqpg\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883389    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-node-log\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.883853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-systemd\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-sys-fs\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.883968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-var-lib-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-run\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-netd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-var-lib-kubelet\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.884701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-systemd-units\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.884978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884705    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 16:19:23.884978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c5441c1-42df-46c1-a4db-f41190a0f2a8-iptables-alerter-script\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.884978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-cni-bin\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885129 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-config\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885129 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-kubelet\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885129 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-modprobe-d\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysctl-conf\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-log-socket\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-etc-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.884970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c5441c1-42df-46c1-a4db-f41190a0f2a8-host-slash\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-sysconfig\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.885260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-openvswitch\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-systemd\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885287    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-serviceca\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-host\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-lib-modules\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-run-netns\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.885709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-env-overrides\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.886015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-host\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.886015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-socket-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.886015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.885997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovnkube-script-lib\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.886153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-kubernetes\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.886153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-run-ovn\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.886153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c064c9c-2559-47fd-a423-d39d6d5f6b25-host-slash\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.886153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886121    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-registration-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.886153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-sys\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.886394 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-konnectivity-ca\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.886954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.886498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54ddde38-4866-430b-a36a-ac60a8230e57-device-dir\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.891001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.890742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" event={"ID":"67dd56507bf34e3ef1ec1fea9b6f36e9","Type":"ContainerStarted","Data":"29473bf4a4d4ff393effbc972b28ec7aad9e14326eeecdf55f0f1688bb67a84e"}
Sep 07 16:19:23.891001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.890790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-etc-tuned\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.891371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.891347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf-agent-certs\") pod \"konnectivity-agent-p5n2k\" (UID: \"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf\") " pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:23.891569 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.891544    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c064c9c-2559-47fd-a423-d39d6d5f6b25-ovn-node-metrics-cert\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.892268 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.892242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-tmp\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.892977 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.892947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"89dcf88f0bd9b87ca70ae75da4c728ad","Type":"ContainerStarted","Data":"3818237894d64fbae791be94f6557afbd479548a6899d5239b6d4d72c02c8fb9"}
Sep 07 16:19:23.905172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.905147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmdmn\" (UniqueName: \"kubernetes.io/projected/54ddde38-4866-430b-a36a-ac60a8230e57-kube-api-access-kmdmn\") pod \"aws-ebs-csi-driver-node-zlqbh\" (UID: \"54ddde38-4866-430b-a36a-ac60a8230e57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:23.905666 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.905618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpj5f\" (UniqueName: \"kubernetes.io/projected/c1746538-b2dc-4bf1-be5e-1bb8fa5bde10-kube-api-access-dpj5f\") pod \"node-ca-xcftb\" (UID: \"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10\") " pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:23.907016 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.906982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnlwl\" (UniqueName: \"kubernetes.io/projected/6c064c9c-2559-47fd-a423-d39d6d5f6b25-kube-api-access-pnlwl\") pod \"ovnkube-node-vsjqp\" (UID: \"6c064c9c-2559-47fd-a423-d39d6d5f6b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:23.907456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.907437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r742v\" (UniqueName: \"kubernetes.io/projected/5dc26d84-cfa2-4609-9bf1-fc9262f2c71c-kube-api-access-r742v\") pod \"tuned-whj92\" (UID: \"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:23.908355 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.908329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp8mz\" (UniqueName: \"kubernetes.io/projected/8c5441c1-42df-46c1-a4db-f41190a0f2a8-kube-api-access-vp8mz\") pod \"iptables-alerter-w5hn5\" (UID: \"8c5441c1-42df-46c1-a4db-f41190a0f2a8\") " pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:23.984132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-multus-certs\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-etc-kubernetes\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-etc-kubernetes\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-multus-certs\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-k8s-cni-cncf-io\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-k8s-cni-cncf-io\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-multus\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-dbus\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.984366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-system-cni-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-conf-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-bin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec32b97f-faca-4bcb-8733-9b95a727a079-hosts-file\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-kubelet-config\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cnibin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-hostroot\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-dbus\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-cnibin\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-multus\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6gl\" (UniqueName: \"kubernetes.io/projected/d81a54d9-b6b8-4822-b998-e823d5ba34f0-kube-api-access-jx6gl\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60c3d654-54ef-47cd-bc10-3f1173214ca8-kubelet-config\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-netns\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.984714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984656    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-system-cni-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-run-netns\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cnibin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-socket-dir-parent\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-hostroot\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984804    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-socket-dir-parent\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.984851    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-cni-bin\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-conf-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.984950    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:24.48490877 +0000 UTC m=+3.129204090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.984995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-system-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-daemon-config\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7lhp\" (UniqueName: \"kubernetes.io/projected/1edc37cf-dc0d-4e09-977f-30a393e7be84-kube-api-access-m7lhp\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec32b97f-faca-4bcb-8733-9b95a727a079-hosts-file\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec32b97f-faca-4bcb-8733-9b95a727a079-tmp-dir\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.985527 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.985125    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.985183    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:24.485167438 +0000 UTC m=+3.129462752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfkg9\" (UniqueName: \"kubernetes.io/projected/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-kube-api-access-tfkg9\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-cnibin\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-kubelet\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvqpg\" (UniqueName: \"kubernetes.io/projected/ec32b97f-faca-4bcb-8733-9b95a727a079-kube-api-access-bvqpg\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cni-binary-copy\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-system-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-host-var-lib-kubelet\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-cni-dir\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985544    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc37cf-dc0d-4e09-977f-30a393e7be84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec32b97f-faca-4bcb-8733-9b95a727a079-tmp-dir\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.985870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc37cf-dc0d-4e09-977f-30a393e7be84-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.986188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.986098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-multus-daemon-config\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.986942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.986222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-cni-binary-copy\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.992326 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.992303    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:23.992443 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.992332    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:23.992443 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.992347    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:23.992443 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:23.992402    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:24.492386568 +0000 UTC m=+3.136681875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:23.995567 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.995522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfkg9\" (UniqueName: \"kubernetes.io/projected/d56d8bb8-cab4-41d9-a30e-4ba23a98c374-kube-api-access-tfkg9\") pod \"multus-6hbb7\" (UID: \"d56d8bb8-cab4-41d9-a30e-4ba23a98c374\") " pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:23.997919 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.997470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7lhp\" (UniqueName: \"kubernetes.io/projected/1edc37cf-dc0d-4e09-977f-30a393e7be84-kube-api-access-m7lhp\") pod \"multus-additional-cni-plugins-2zdjd\" (UID: \"1edc37cf-dc0d-4e09-977f-30a393e7be84\") " pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:23.997919 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.997536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvqpg\" (UniqueName: \"kubernetes.io/projected/ec32b97f-faca-4bcb-8733-9b95a727a079-kube-api-access-bvqpg\") pod \"node-resolver-zhl2d\" (UID: \"ec32b97f-faca-4bcb-8733-9b95a727a079\") " pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:23.997919 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:23.997887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6gl\" (UniqueName: \"kubernetes.io/projected/d81a54d9-b6b8-4822-b998-e823d5ba34f0-kube-api-access-jx6gl\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:24.071850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.071738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-whj92"
Sep 07 16:19:24.084657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.084628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w5hn5"
Sep 07 16:19:24.094952 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.094928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:24.104041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.104018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:24.114634 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.114613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xcftb"
Sep 07 16:19:24.120814 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.120789    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh"
Sep 07 16:19:24.127841 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.127818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zhl2d"
Sep 07 16:19:24.135893 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.135868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2zdjd"
Sep 07 16:19:24.142839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.142819    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hbb7"
Sep 07 16:19:24.489433 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.489340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:24.489433 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.489394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:24.489675 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.489537    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:24.489675 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.489559    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:24.489675 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.489612    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:25.489591181 +0000 UTC m=+4.133886470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:24.489675 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.489635    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:25.48962299 +0000 UTC m=+4.133918286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:24.590271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.590220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:24.590455 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.590422    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:24.590455 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.590449    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:24.590559 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.590465    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:24.590559 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.590551    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:25.590535102 +0000 UTC m=+4.234830391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:24.812540 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.812408    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 16:14:23 +0000 UTC" deadline="2028-02-17 04:02:17.090670368 +0000 UTC"
Sep 07 16:19:24.812540 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.812494    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12659h42m52.278181752s"
Sep 07 16:19:24.882456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:24.882421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:24.882647 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:24.882541    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:25.090986 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:25.090953    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c064c9c_2559_47fd_a423_d39d6d5f6b25.slice/crio-89d237d782a170db9ac9732c30daeb215ca457b055d23cc1d36cc02a2ba09b7b WatchSource:0}: Error finding container 89d237d782a170db9ac9732c30daeb215ca457b055d23cc1d36cc02a2ba09b7b: Status 404 returned error can't find the container with id 89d237d782a170db9ac9732c30daeb215ca457b055d23cc1d36cc02a2ba09b7b
Sep 07 16:19:25.093865 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:25.093595    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5441c1_42df_46c1_a4db_f41190a0f2a8.slice/crio-bf6d55824be6a785bdd8ca0a99e8f30ca57fa608d3bf1bcd6cd2f1c39a33a9dd WatchSource:0}: Error finding container bf6d55824be6a785bdd8ca0a99e8f30ca57fa608d3bf1bcd6cd2f1c39a33a9dd: Status 404 returned error can't find the container with id bf6d55824be6a785bdd8ca0a99e8f30ca57fa608d3bf1bcd6cd2f1c39a33a9dd
Sep 07 16:19:25.094658 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:25.094617    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edc37cf_dc0d_4e09_977f_30a393e7be84.slice/crio-ac581c97a85dfdac202b01080b33bb3293be263130842c01f2ed56194d1d9ad7 WatchSource:0}: Error finding container ac581c97a85dfdac202b01080b33bb3293be263130842c01f2ed56194d1d9ad7: Status 404 returned error can't find the container with id ac581c97a85dfdac202b01080b33bb3293be263130842c01f2ed56194d1d9ad7
Sep 07 16:19:25.099752 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:25.099217    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56d8bb8_cab4_41d9_a30e_4ba23a98c374.slice/crio-e6d3dc9f99eaf37ccc2cbf928df11972ce49a2c665e767f504860ef83fec8a2c WatchSource:0}: Error finding container e6d3dc9f99eaf37ccc2cbf928df11972ce49a2c665e767f504860ef83fec8a2c: Status 404 returned error can't find the container with id e6d3dc9f99eaf37ccc2cbf928df11972ce49a2c665e767f504860ef83fec8a2c
Sep 07 16:19:25.497238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.497197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:25.497238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.497238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:25.497457 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.497376    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:25.497457 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.497382    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:25.497457 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.497442    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:27.497426234 +0000 UTC m=+6.141721522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:25.497457 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.497459    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:27.497450156 +0000 UTC m=+6.141745444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:25.598114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.598024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:25.598262 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.598193    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:25.598262 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.598223    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:25.598262 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.598237    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:25.598383 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.598309    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:27.598287158 +0000 UTC m=+6.242582451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:25.813238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.813190    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 16:14:23 +0000 UTC" deadline="2028-05-25 17:45:03.568299698 +0000 UTC"
Sep 07 16:19:25.813238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.813236    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15025h25m37.755069192s"
Sep 07 16:19:25.883045 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.882890    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:25.883203 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.883039    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:25.883506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.883336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:25.883506 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:25.883463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:25.902483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.902441    2579 generic.go:356] "Generic (PLEG): container finished" podID="89dcf88f0bd9b87ca70ae75da4c728ad" containerID="826a804db29fedb87939e215edad99051c44415e7553cd989add254c9c7f73ab" exitCode=0
Sep 07 16:19:25.902657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.902593    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"89dcf88f0bd9b87ca70ae75da4c728ad","Type":"ContainerDied","Data":"826a804db29fedb87939e215edad99051c44415e7553cd989add254c9c7f73ab"}
Sep 07 16:19:25.911228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.911191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" event={"ID":"67dd56507bf34e3ef1ec1fea9b6f36e9","Type":"ContainerStarted","Data":"4d8e6c747ae2739015b5920e6daf7ebb2848015a100b87fa3faba2a571ebdc6f"}
Sep 07 16:19:25.917342 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.917152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh" event={"ID":"54ddde38-4866-430b-a36a-ac60a8230e57","Type":"ContainerStarted","Data":"584db867f2e1ad505fe0d075bcae507ae227b0f15285fc34ce34ed900eb89946"}
Sep 07 16:19:25.931246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.930349    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-154.ec2.internal" podStartSLOduration=2.930328186 podStartE2EDuration="2.930328186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:25.929231765 +0000 UTC m=+4.573527077" watchObservedRunningTime="2026-09-07 16:19:25.930328186 +0000 UTC m=+4.574623501"
Sep 07 16:19:25.933090 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.933057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p5n2k" event={"ID":"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf","Type":"ContainerStarted","Data":"6b192fca541cdd22da4763470ae6dbe3b5ec24d6614f141f1e52569d944a9855"}
Sep 07 16:19:25.948841 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.948765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-whj92" event={"ID":"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c","Type":"ContainerStarted","Data":"9f9efedfa519f95b0f25da145cce954ae05e45c45c83e25f58bd787fcb1cfbdf"}
Sep 07 16:19:25.959489 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.959403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerStarted","Data":"ac581c97a85dfdac202b01080b33bb3293be263130842c01f2ed56194d1d9ad7"}
Sep 07 16:19:25.977956 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.977903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"89d237d782a170db9ac9732c30daeb215ca457b055d23cc1d36cc02a2ba09b7b"}
Sep 07 16:19:25.993124 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:25.993008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zhl2d" event={"ID":"ec32b97f-faca-4bcb-8733-9b95a727a079","Type":"ContainerStarted","Data":"9ec3f71f00c9e2269392f4e397c89fa7cf5fdf0336fa9ee1e803335401eec048"}
Sep 07 16:19:26.009467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:26.009354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xcftb" event={"ID":"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10","Type":"ContainerStarted","Data":"9cc78ec09513c3cf76d396113264c7b1099cacfcfd0365b49453b6366f505585"}
Sep 07 16:19:26.025165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:26.025119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hbb7" event={"ID":"d56d8bb8-cab4-41d9-a30e-4ba23a98c374","Type":"ContainerStarted","Data":"e6d3dc9f99eaf37ccc2cbf928df11972ce49a2c665e767f504860ef83fec8a2c"}
Sep 07 16:19:26.028327 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:26.028293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w5hn5" event={"ID":"8c5441c1-42df-46c1-a4db-f41190a0f2a8","Type":"ContainerStarted","Data":"bf6d55824be6a785bdd8ca0a99e8f30ca57fa608d3bf1bcd6cd2f1c39a33a9dd"}
Sep 07 16:19:26.882001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:26.881965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:26.882464 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:26.882115    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:27.045846 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.045808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" event={"ID":"89dcf88f0bd9b87ca70ae75da4c728ad","Type":"ContainerStarted","Data":"f9e3ba6dc0bfc7c8c2704ecf9cf0f46a34b7283c12af55969149f8e5231265b4"}
Sep 07 16:19:27.522026 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.521985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:27.522213 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.522098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:27.522213 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.522167    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:27.522213 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.522206    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:27.522364 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.522245    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:31.522223128 +0000 UTC m=+10.166518421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:27.522364 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.522264    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:31.522254703 +0000 UTC m=+10.166549992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:27.622866 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.622826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:27.623049 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.623030    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:27.623123 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.623054    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:27.623123 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.623070    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:27.623215 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.623127    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:31.623110296 +0000 UTC m=+10.267405597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:27.884805 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.884710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:27.885229 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.884860    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:27.885344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:27.885325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:27.885452 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:27.885430    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:28.882337 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:28.881815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:28.882337 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:28.881968    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:29.883360 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:29.883327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:29.883813 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:29.883327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:29.883813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:29.883437    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:29.883813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:29.883539    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:30.882382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:30.882344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:30.882540 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:30.882463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:31.555120 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:31.555083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:31.555562 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:31.555129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:31.555562 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.555272    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:31.555562 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.555334    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:39.555313642 +0000 UTC m=+18.199608993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:31.555699 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.555631    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:31.555699 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.555690    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:39.555663845 +0000 UTC m=+18.199959138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:31.656546 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:31.656507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:31.656737 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.656655    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:31.656737 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.656678    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:31.656737 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.656690    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:31.656905 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.656766    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:39.656730264 +0000 UTC m=+18.301025557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:31.883194 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:31.882874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:31.883344 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.883237    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:31.883581 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:31.882940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:31.883683 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:31.883661    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:32.881590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:32.881561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:32.882072 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:32.881693    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:33.882243 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:33.882204    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:33.882665 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:33.882335    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:33.882782 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:33.882761    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:33.882904 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:33.882880    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:34.882058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:34.882019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:34.882239 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:34.882151    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:35.882177 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:35.882138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:35.882556 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:35.882246    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:35.882556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:35.882277    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:35.882556 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:35.882327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:36.881583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:36.881551    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:36.881806 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:36.881683    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:37.882272 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:37.882237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:37.882732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:37.882236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:37.882732 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:37.882387    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:37.882732 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:37.882432    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:38.882305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:38.882272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:38.882835 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:38.882418    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:39.617554 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:39.617520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:39.617554 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:39.617559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:39.617839 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.617662    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:39.617839 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.617675    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:39.617839 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.617717    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:55.617702914 +0000 UTC m=+34.261998202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:39.617839 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.617740    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:55.617722608 +0000 UTC m=+34.262017915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:39.718496 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:39.718454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:39.718651 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.718617    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:39.718651 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.718636    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:39.718651 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.718646    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:39.718767 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.718699    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:55.718684785 +0000 UTC m=+34.362980074 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:39.881774 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:39.881669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:39.881985 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:39.881676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:39.881985 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.881844    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:39.881985 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:39.881912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:40.881591 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:40.881558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:40.882038 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:40.881690    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:41.882818 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:41.882782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:41.883255 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:41.882903    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:41.883323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:41.883261    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:41.883372 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:41.883346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:42.881698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:42.881655    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:42.881915 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:42.881841    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:43.883059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:43.882860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:43.884039 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:43.882863    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:43.884039 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:43.883180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:43.884039 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:43.883606    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:44.088586 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.088544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh" event={"ID":"54ddde38-4866-430b-a36a-ac60a8230e57","Type":"ContainerStarted","Data":"6ca3567f732329e7eb984b9f5072f7d32e189c3fe4008bd53b602dcb47cd36ea"}
Sep 07 16:19:44.089993 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.089964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p5n2k" event={"ID":"5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf","Type":"ContainerStarted","Data":"3048c353781f08245c6d626252371b035ef95103c2c4cf5e0a164ef171c6f304"}
Sep 07 16:19:44.091567 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.091537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-whj92" event={"ID":"5dc26d84-cfa2-4609-9bf1-fc9262f2c71c","Type":"ContainerStarted","Data":"e19f21cfcd9961d0388f8282db5098b2f2a8f0530ace1fdaea21cc3a270fc240"}
Sep 07 16:19:44.093120 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.093095    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="6c3cb6f206385f6b089bb829f760884e021c58f62db58759dbc7bdde225ac7c6" exitCode=0
Sep 07 16:19:44.093225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.093147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"6c3cb6f206385f6b089bb829f760884e021c58f62db58759dbc7bdde225ac7c6"}
Sep 07 16:19:44.098652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"f7206f268ec656831327d7a06810f2e87fd6efdfd8d8c78d7bb18791f286b25a"}
Sep 07 16:19:44.098770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"853bbe02969d0e9f70dba854718ece822ccb58922083fa1744d1fed1f082230e"}
Sep 07 16:19:44.098770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"f6c4b66b11554b7757290356ee7c1b6ffd9f7938b5b527501a0c96ea9f75e80f"}
Sep 07 16:19:44.098880 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"a0d756110d2640a4b391f9dc15215d2d333792cc36079334497e86f37db3092d"}
Sep 07 16:19:44.098880 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"310f1e68381bed84674c8cc4c8ea579792eb5d0db3070baff09a610667a546cf"}
Sep 07 16:19:44.098880 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.098863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"ea9dc95048774a6cad16372a3a268cbe9c1d82fbf5fb10c72eecb56e2e429061"}
Sep 07 16:19:44.100090 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.100071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zhl2d" event={"ID":"ec32b97f-faca-4bcb-8733-9b95a727a079","Type":"ContainerStarted","Data":"8145347ec4476d7357fe447257f5781fa11f82051787732646429281609f0970"}
Sep 07 16:19:44.102249 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.102221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xcftb" event={"ID":"c1746538-b2dc-4bf1-be5e-1bb8fa5bde10","Type":"ContainerStarted","Data":"bb9ab0d76d164b03f138971d02b9aa3fbf8eb22ad099a8a05bd981c56b85a0a3"}
Sep 07 16:19:44.104133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.104106    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hbb7" event={"ID":"d56d8bb8-cab4-41d9-a30e-4ba23a98c374","Type":"ContainerStarted","Data":"571aa2eb4442fe90f1eb20e5a4c3e0d9be0f478f8605c6d61e4fa8a5bfacb784"}
Sep 07 16:19:44.104583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.104547    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-154.ec2.internal" podStartSLOduration=21.104535556 podStartE2EDuration="21.104535556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:27.058782632 +0000 UTC m=+5.703077960" watchObservedRunningTime="2026-09-07 16:19:44.104535556 +0000 UTC m=+22.748830867"
Sep 07 16:19:44.115770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.115714    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p5n2k" podStartSLOduration=4.075614842 podStartE2EDuration="22.115700914s" totalImagesPullingTime="18.040086072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:44.104366425 +0000 UTC m=+22.748661760" watchObservedRunningTime="2026-09-07 16:19:44.115700914 +0000 UTC m=+22.759996221"
Sep 07 16:19:44.116233 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.116196    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zhl2d" podStartSLOduration=4.076704121 podStartE2EDuration="22.11618436s" totalImagesPullingTime="18.039480239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:44.115940075 +0000 UTC m=+22.760235406" watchObservedRunningTime="2026-09-07 16:19:44.11618436 +0000 UTC m=+22.760479670"
Sep 07 16:19:44.146957 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.146893    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6hbb7" podStartSLOduration=4.032197487 podStartE2EDuration="22.146874473s" totalImagesPullingTime="18.114676986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:44.146506042 +0000 UTC m=+22.790801372" watchObservedRunningTime="2026-09-07 16:19:44.146874473 +0000 UTC m=+22.791169787"
Sep 07 16:19:44.161106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.161055    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-whj92" podStartSLOduration=4.095118525 podStartE2EDuration="22.161038287s" totalImagesPullingTime="18.065919762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:44.160950844 +0000 UTC m=+22.805246173" watchObservedRunningTime="2026-09-07 16:19:44.161038287 +0000 UTC m=+22.805333598"
Sep 07 16:19:44.173125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.173077    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xcftb" podStartSLOduration=4.125345603 podStartE2EDuration="22.17306493s" totalImagesPullingTime="18.047719327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:44.172661253 +0000 UTC m=+22.816956564" watchObservedRunningTime="2026-09-07 16:19:44.17306493 +0000 UTC m=+22.817360273"
Sep 07 16:19:44.730933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.730904    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 16:19:44.839969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.839804    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T16:19:44.73092856Z","UUID":"541a7f98-584d-40c6-99a6-db56a2408543","Handler":null,"Name":"","Endpoint":""}
Sep 07 16:19:44.841557 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.841535    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 16:19:44.841689 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.841572    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 16:19:44.881984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:44.881961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:44.882148 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:44.882071    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:45.109792 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.109676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh" event={"ID":"54ddde38-4866-430b-a36a-ac60a8230e57","Type":"ContainerStarted","Data":"36d3bb65e3dff3400b561b5d8e061f83b39dd6d2db89601ca4c0b77b2073f623"}
Sep 07 16:19:45.111529 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.111498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w5hn5" event={"ID":"8c5441c1-42df-46c1-a4db-f41190a0f2a8","Type":"ContainerStarted","Data":"8d9eeb6d279ff063da98ae471b4d3521c71d1cd188727aab7f5b8601de749dac"}
Sep 07 16:19:45.124679 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.124627    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w5hn5" podStartSLOduration=5.068279451 podStartE2EDuration="23.12460915s" totalImagesPullingTime="18.056329699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:45.124294332 +0000 UTC m=+23.768589645" watchObservedRunningTime="2026-09-07 16:19:45.12460915 +0000 UTC m=+23.768904460"
Sep 07 16:19:45.534056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.533821    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:45.882014 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.881984    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:45.882014 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:45.881983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:45.882199 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:45.882171    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:45.882372 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:45.882316    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:46.116091 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:46.116054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"6baf6b2f5e32a5278c59d2ed01a5c08cd1c7934367aba630553c5dae3128b932"}
Sep 07 16:19:46.882137 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:46.882108    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:46.882322 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:46.882215    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:47.120343 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.120309    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh" event={"ID":"54ddde38-4866-430b-a36a-ac60a8230e57","Type":"ContainerStarted","Data":"c39d1f1485906025230980ce7759f6b6f01ac3b101e34ec3e3d6262a83af5300"}
Sep 07 16:19:47.135344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.135260    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zlqbh" podStartSLOduration=4.594916349 podStartE2EDuration="25.135247345s" totalImagesPullingTime="20.540330996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:47.134674052 +0000 UTC m=+25.778969362" watchObservedRunningTime="2026-09-07 16:19:47.135247345 +0000 UTC m=+25.779542654"
Sep 07 16:19:47.211155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.211121    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:47.211920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.211899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:47.881916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.881410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:47.881916 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:47.881532    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:47.881916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:47.881591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:47.881916 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:47.881691    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:48.123081 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:48.123051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p5n2k"
Sep 07 16:19:48.881612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:48.881583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:48.881803 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:48.881689    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:49.128361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.128320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" event={"ID":"6c064c9c-2559-47fd-a423-d39d6d5f6b25","Type":"ContainerStarted","Data":"015fa41e80363246eca2410c15d583082da605d1ce2884bb9d5b2ebd5fdaab49"}
Sep 07 16:19:49.128793 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.128601    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:49.128793 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.128680    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:49.144892 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.144827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:49.152153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.152108    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp" podStartSLOduration=9.026275258 podStartE2EDuration="27.15209336s" totalImagesPullingTime="18.125818102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:19:49.151588117 +0000 UTC m=+27.795883427" watchObservedRunningTime="2026-09-07 16:19:49.15209336 +0000 UTC m=+27.796388670"
Sep 07 16:19:49.882381 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.882169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:49.882554 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:49.882207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:49.882554 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:49.882478    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:49.882554 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:49.882523    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:50.131965 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.131932    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="ebaa23d99b47e6568c99238be6afd4a6a1d2bffef9d82145ab0fb221c9e9c7ca" exitCode=0
Sep 07 16:19:50.132414 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.132024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"ebaa23d99b47e6568c99238be6afd4a6a1d2bffef9d82145ab0fb221c9e9c7ca"}
Sep 07 16:19:50.132539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.132523    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:50.148825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.148797    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:19:50.860425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.860393    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89lj5"]
Sep 07 16:19:50.860626 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.860536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:50.860701 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:50.860626    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:50.864219 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.864195    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pc54l"]
Sep 07 16:19:50.864359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.864304    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:50.864451 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:50.864412    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:50.864969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.864952    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zj7m6"]
Sep 07 16:19:50.865059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:50.865048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:50.865154 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:50.865131    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:52.139982 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:52.139946    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="81c8a6390f5e5005d73272d7f20a005f74e4354847f3f4e67960e5f7f40f9e54" exitCode=0
Sep 07 16:19:52.140826 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:52.140036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"81c8a6390f5e5005d73272d7f20a005f74e4354847f3f4e67960e5f7f40f9e54"}
Sep 07 16:19:52.882635 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:52.882392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:52.882796 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:52.882415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:52.882796 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:52.882678    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:52.882796 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:52.882420    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:52.885402 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:52.883102    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:52.885402 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:52.883255    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:54.146253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:54.146216    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="248e57ffaedffcc9b7e5e72ee34ec2e1adfb043aa60186d82e07c021d3d88101" exitCode=0
Sep 07 16:19:54.146616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:54.146281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"248e57ffaedffcc9b7e5e72ee34ec2e1adfb043aa60186d82e07c021d3d88101"}
Sep 07 16:19:54.882177 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:54.882097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:54.882177 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:54.882141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:54.882365 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:54.882187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:54.882365 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:54.882203    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pc54l" podUID="6456325c-0cf2-4af9-b979-d3cd149e8014"
Sep 07 16:19:54.882365 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:54.882261    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zj7m6" podUID="d81a54d9-b6b8-4822-b998-e823d5ba34f0"
Sep 07 16:19:54.882365 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:54.882310    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89lj5" podUID="60c3d654-54ef-47cd-bc10-3f1173214ca8"
Sep 07 16:19:55.631250 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:55.631212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:55.631717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:55.631330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:55.631717 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.631416    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:55.631717 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.631450    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:55.631717 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.631499    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret podName:60c3d654-54ef-47cd-bc10-3f1173214ca8 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:27.631477575 +0000 UTC m=+66.275772881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret") pod "global-pull-secret-syncer-89lj5" (UID: "60c3d654-54ef-47cd-bc10-3f1173214ca8") : object "kube-system"/"original-pull-secret" not registered
Sep 07 16:19:55.631717 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.631522    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs podName:d81a54d9-b6b8-4822-b998-e823d5ba34f0 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:27.63151132 +0000 UTC m=+66.275806612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs") pod "network-metrics-daemon-zj7m6" (UID: "d81a54d9-b6b8-4822-b998-e823d5ba34f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 16:19:55.731872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:55.731841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:55.732020 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.731980    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 16:19:55.732020 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.731997    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 16:19:55.732020 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.732010    2579 projected.go:196] Error preparing data for projected volume kube-api-access-t6kws for pod openshift-network-diagnostics/network-check-target-pc54l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:55.732114 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:55.732061    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws podName:6456325c-0cf2-4af9-b979-d3cd149e8014 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:27.732048496 +0000 UTC m=+66.376343784 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6kws" (UniqueName: "kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws") pod "network-check-target-pc54l" (UID: "6456325c-0cf2-4af9-b979-d3cd149e8014") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 16:19:56.175575 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.175548    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-154.ec2.internal" event="NodeReady"
Sep 07 16:19:56.175724 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.175671    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 16:19:56.207787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.207690    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"]
Sep 07 16:19:56.211787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.211762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.214846 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.213582    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:19:56.215301 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.215281    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.215629 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.215604    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.217048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.216120    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 07 16:19:56.219059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.218840    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pg7bg\""
Sep 07 16:19:56.219167 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.219117    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 07 16:19:56.222370 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.222349    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ncq5c"]
Sep 07 16:19:56.222501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.222477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.224715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.224701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 16:19:56.225885 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.225867    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bxjnn"]
Sep 07 16:19:56.226025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.225971    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.226313 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.226292    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qt7fz\""
Sep 07 16:19:56.226894 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.226877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 16:19:56.227028 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.226925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 16:19:56.229424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.229406    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ljt68\""
Sep 07 16:19:56.229652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.229462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 07 16:19:56.229652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.229484    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.229652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.229486    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.229652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.229628    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 07 16:19:56.230162 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.230142    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"]
Sep 07 16:19:56.232117 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.231226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.238246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.237510    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8688c84cc4-cltmt"]
Sep 07 16:19:56.238246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.237632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"
Sep 07 16:19:56.242536 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.242300    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"]
Sep 07 16:19:56.243013 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.242996    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 07 16:19:56.243154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.243132    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 07 16:19:56.243300 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.242996    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.243479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.243391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 16:19:56.243479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.243410    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2qlx7\""
Sep 07 16:19:56.243479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.243446    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.243661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.243604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.244466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.244448    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.244640 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.244455    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.245889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.245285    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-28x2c\""
Sep 07 16:19:56.245889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.245631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.247376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247353    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"]
Sep 07 16:19:56.247483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247400    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"]
Sep 07 16:19:56.247483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"]
Sep 07 16:19:56.247483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bxjnn"]
Sep 07 16:19:56.247483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247356    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rdllp\""
Sep 07 16:19:56.247725 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.248008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.247981    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 07 16:19:56.249589 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.249571    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 07 16:19:56.249984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.249965    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.250186 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.250167    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t2gln\""
Sep 07 16:19:56.250346 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.250331    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ncq5c"]
Sep 07 16:19:56.250461 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.250441    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8688c84cc4-cltmt"]
Sep 07 16:19:56.250639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.250624    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.250826 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.250811    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 07 16:19:56.251443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.251426    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 07 16:19:56.251929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.251705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 07 16:19:56.252131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.252116    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.252866 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.252846    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 07 16:19:56.255245 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.254337    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r9h7m"]
Sep 07 16:19:56.259133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.259107    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 07 16:19:56.262075 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.262055    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"]
Sep 07 16:19:56.263891 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.263870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.266189 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:19:56.266285 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266198    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jr97m"]
Sep 07 16:19:56.266852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.267013 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266870    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 16:19:56.267082 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266903    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c2wmp\""
Sep 07 16:19:56.267132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.266924    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 16:19:56.269206 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.269184    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 07 16:19:56.269534 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.269457    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.269689 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.269658    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 07 16:19:56.270124 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.270103    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gqclh"]
Sep 07 16:19:56.270319 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.270212    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cclw5\""
Sep 07 16:19:56.270441 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.270417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.271049 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.270116    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.272422 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.272403    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.273003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.272988    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 16:19:56.273121 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.273019    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jt2mt\""
Sep 07 16:19:56.273233 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.273055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.273381 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.273355    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"]
Sep 07 16:19:56.273557 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.273412    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh"
Sep 07 16:19:56.275580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.275560    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.275801 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.275785    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.275953 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.275936    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r269p\""
Sep 07 16:19:56.276852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.276833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr97m"]
Sep 07 16:19:56.276983 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.276966    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"]
Sep 07 16:19:56.277111 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.276987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gqclh"]
Sep 07 16:19:56.277111 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.277004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"]
Sep 07 16:19:56.277111 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.277027    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9h7m"]
Sep 07 16:19:56.277111 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.276857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.278965 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.278944    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 07 16:19:56.279144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.279122    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:19:56.279144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.279138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 07 16:19:56.279386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.279370    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zkxj8\""
Sep 07 16:19:56.279386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.279378    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 07 16:19:56.319703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.319681    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qc4gc"]
Sep 07 16:19:56.324567 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.324548    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.326847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.326830    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7chqs\""
Sep 07 16:19:56.327006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.326875    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 07 16:19:56.327153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.326879    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 07 16:19:56.331086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.331057    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qc4gc"]
Sep 07 16:19:56.336187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-snapshots\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.336297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.336297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-stats-auth\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.336297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2ksf\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.336428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj9br\" (UniqueName: \"kubernetes.io/projected/fce3af34-7b9b-4b7d-a3a7-d182414b5330-kube-api-access-rj9br\") pod \"network-check-source-685456655f-gqclh\" (UID: \"fce3af34-7b9b-4b7d-a3a7-d182414b5330\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh"
Sep 07 16:19:56.336428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.336428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-serving-cert\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.336428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7cbfe08-e923-4162-b279-2f1514acff96-cert\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pndnz\" (UniqueName: \"kubernetes.io/projected/a7cbfe08-e923-4162-b279-2f1514acff96-kube-api-access-pndnz\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdc79fad-e548-465c-ab98-afaefef818d5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc79fad-e548-465c-ab98-afaefef818d5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.336619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29hbr\" (UniqueName: \"kubernetes.io/projected/fdc79fad-e548-465c-ab98-afaefef818d5-kube-api-access-29hbr\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jx8\" (UniqueName: \"kubernetes.io/projected/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-kube-api-access-b2jx8\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dd7487-9e46-4d8b-aba1-7d208e5866fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336715    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4cz\" (UniqueName: \"kubernetes.io/projected/61dd7487-9e46-4d8b-aba1-7d208e5866fc-kube-api-access-bj4cz\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3d62372-86a4-46e4-bb1e-9a4622096b13-serving-cert\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6adcc7c9-5071-43f7-8179-31900ce7f8c2-config-volume\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd5qg\" (UniqueName: \"kubernetes.io/projected/fcd813e3-85db-47fc-8d80-d172471725c9-kube-api-access-nd5qg\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6adcc7c9-5071-43f7-8179-31900ce7f8c2-tmp-dir\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.336948 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-config\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.336980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/de9dc804-3b56-4941-8d7f-dae66f7ffd14-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-config\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.337417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcx8v\" (UniqueName: \"kubernetes.io/projected/6adcc7c9-5071-43f7-8179-31900ce7f8c2-kube-api-access-fcx8v\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.337599 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-trusted-ca\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.337599 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.337660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdww\" (UniqueName: \"kubernetes.io/projected/d3d62372-86a4-46e4-bb1e-9a4622096b13-kube-api-access-9mdww\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.337660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.337660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-tmp\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.337787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.337787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-default-certificate\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.337787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpq8n\" (UniqueName: \"kubernetes.io/projected/de9dc804-3b56-4941-8d7f-dae66f7ffd14-kube-api-access-qpq8n\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.337884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmmf\" (UniqueName: \"kubernetes.io/projected/7a90ede4-bd7c-4b11-8144-1d87d18fe69e-kube-api-access-qsmmf\") pod \"volume-data-source-validator-695c895bd6-lrbmf\" (UID: \"7a90ede4-bd7c-4b11-8144-1d87d18fe69e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"
Sep 07 16:19:56.337884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337827    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.337884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7gc2\" (UniqueName: \"kubernetes.io/projected/d86f2ea3-df9d-4ce5-af14-0e49782f295a-kube-api-access-t7gc2\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.337884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.337873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.438773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.438707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-stats-auth\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.439071 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2ksf\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.439168 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj9br\" (UniqueName: \"kubernetes.io/projected/fce3af34-7b9b-4b7d-a3a7-d182414b5330-kube-api-access-rj9br\") pod \"network-check-source-685456655f-gqclh\" (UID: \"fce3af34-7b9b-4b7d-a3a7-d182414b5330\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh"
Sep 07 16:19:56.439168 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.439168 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-serving-cert\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7cbfe08-e923-4162-b279-2f1514acff96-cert\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pndnz\" (UniqueName: \"kubernetes.io/projected/a7cbfe08-e923-4162-b279-2f1514acff96-kube-api-access-pndnz\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdc79fad-e548-465c-ab98-afaefef818d5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.439323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc79fad-e548-465c-ab98-afaefef818d5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hbr\" (UniqueName: \"kubernetes.io/projected/fdc79fad-e548-465c-ab98-afaefef818d5-kube-api-access-29hbr\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jx8\" (UniqueName: \"kubernetes.io/projected/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-kube-api-access-b2jx8\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dd7487-9e46-4d8b-aba1-7d208e5866fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4cz\" (UniqueName: \"kubernetes.io/projected/61dd7487-9e46-4d8b-aba1-7d208e5866fc-kube-api-access-bj4cz\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3d62372-86a4-46e4-bb1e-9a4622096b13-serving-cert\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6adcc7c9-5071-43f7-8179-31900ce7f8c2-config-volume\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd5qg\" (UniqueName: \"kubernetes.io/projected/fcd813e3-85db-47fc-8d80-d172471725c9-kube-api-access-nd5qg\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.439600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.439605    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6adcc7c9-5071-43f7-8179-31900ce7f8c2-tmp-dir\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.439683    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.939667054 +0000 UTC m=+35.583962345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-config\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/de9dc804-3b56-4941-8d7f-dae66f7ffd14-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.439996    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-config\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.440048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2aa09c03-bffd-4df4-99b1-3b80887591cc-nginx-conf\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.440549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcx8v\" (UniqueName: \"kubernetes.io/projected/6adcc7c9-5071-43f7-8179-31900ce7f8c2-kube-api-access-fcx8v\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.440549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.440549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-trusted-ca\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.440549 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.440420    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:19:56.440549 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.440479    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.940462404 +0000 UTC m=+35.584757696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:19:56.440872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.440872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.440861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-trusted-ca\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.441060 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.441039    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6adcc7c9-5071-43f7-8179-31900ce7f8c2-config-volume\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.441533 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.441513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-config\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.441721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc79fad-e548-465c-ab98-afaefef818d5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.441920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3d62372-86a4-46e4-bb1e-9a4622096b13-config\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.442020    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.442035    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.442095    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.942079104 +0000 UTC m=+35.586374394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6adcc7c9-5071-43f7-8179-31900ce7f8c2-tmp-dir\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdww\" (UniqueName: \"kubernetes.io/projected/d3d62372-86a4-46e4-bb1e-9a4622096b13-kube-api-access-9mdww\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-tmp\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-default-certificate\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpq8n\" (UniqueName: \"kubernetes.io/projected/de9dc804-3b56-4941-8d7f-dae66f7ffd14-kube-api-access-qpq8n\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmmf\" (UniqueName: \"kubernetes.io/projected/7a90ede4-bd7c-4b11-8144-1d87d18fe69e-kube-api-access-qsmmf\") pod \"volume-data-source-validator-695c895bd6-lrbmf\" (UID: \"7a90ede4-bd7c-4b11-8144-1d87d18fe69e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"
Sep 07 16:19:56.442789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7gc2\" (UniqueName: \"kubernetes.io/projected/d86f2ea3-df9d-4ce5-af14-0e49782f295a-kube-api-access-t7gc2\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-snapshots\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.442914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/de9dc804-3b56-4941-8d7f-dae66f7ffd14-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.443045    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.943027755 +0000 UTC m=+35.587323062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.443127    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.443162    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.94315113 +0000 UTC m=+35.587446421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:19:56.443565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.443433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-tmp\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.444017 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.443741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61dd7487-9e46-4d8b-aba1-7d208e5866fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.444238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.444149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-serving-cert\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.444238 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.444209    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:56.444344 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.444241    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:56.944229194 +0000 UTC m=+35.588524481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:56.444559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.444527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-stats-auth\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.444786 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.444764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdc79fad-e548-465c-ab98-afaefef818d5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.444990 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.444965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.445344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.445298    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61dd7487-9e46-4d8b-aba1-7d208e5866fc-snapshots\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.445614 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.445402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7cbfe08-e923-4162-b279-2f1514acff96-cert\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.446318 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.446286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.447041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.447018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-default-certificate\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.447339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.447313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3d62372-86a4-46e4-bb1e-9a4622096b13-serving-cert\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.447810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.447726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.448893 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.448839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dd7487-9e46-4d8b-aba1-7d208e5866fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.450085 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.449642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2ksf\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.451424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.451381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd5qg\" (UniqueName: \"kubernetes.io/projected/fcd813e3-85db-47fc-8d80-d172471725c9-kube-api-access-nd5qg\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.451520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.451456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pndnz\" (UniqueName: \"kubernetes.io/projected/a7cbfe08-e923-4162-b279-2f1514acff96-kube-api-access-pndnz\") pod \"ingress-canary-jr97m\" (UID: \"a7cbfe08-e923-4162-b279-2f1514acff96\") " pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.452305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.452279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcx8v\" (UniqueName: \"kubernetes.io/projected/6adcc7c9-5071-43f7-8179-31900ce7f8c2-kube-api-access-fcx8v\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.452789 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.452719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29hbr\" (UniqueName: \"kubernetes.io/projected/fdc79fad-e548-465c-ab98-afaefef818d5-kube-api-access-29hbr\") pod \"kube-storage-version-migrator-operator-8d7dd4675-lstgg\" (UID: \"fdc79fad-e548-465c-ab98-afaefef818d5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.453662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.453083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpq8n\" (UniqueName: \"kubernetes.io/projected/de9dc804-3b56-4941-8d7f-dae66f7ffd14-kube-api-access-qpq8n\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.453662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.453618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj9br\" (UniqueName: \"kubernetes.io/projected/fce3af34-7b9b-4b7d-a3a7-d182414b5330-kube-api-access-rj9br\") pod \"network-check-source-685456655f-gqclh\" (UID: \"fce3af34-7b9b-4b7d-a3a7-d182414b5330\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh"
Sep 07 16:19:56.454178 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.454156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.455267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.455233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jx8\" (UniqueName: \"kubernetes.io/projected/9fbd4161-cad8-431c-9de9-a21c2fcd6fff-kube-api-access-b2jx8\") pod \"service-ca-operator-859f94c4b-bsblw\" (UID: \"9fbd4161-cad8-431c-9de9-a21c2fcd6fff\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.455364 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.455344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdww\" (UniqueName: \"kubernetes.io/projected/d3d62372-86a4-46e4-bb1e-9a4622096b13-kube-api-access-9mdww\") pod \"console-operator-85b454874b-bxjnn\" (UID: \"d3d62372-86a4-46e4-bb1e-9a4622096b13\") " pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.455445 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.455425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7gc2\" (UniqueName: \"kubernetes.io/projected/d86f2ea3-df9d-4ce5-af14-0e49782f295a-kube-api-access-t7gc2\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.456290 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.456269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmmf\" (UniqueName: \"kubernetes.io/projected/7a90ede4-bd7c-4b11-8144-1d87d18fe69e-kube-api-access-qsmmf\") pod \"volume-data-source-validator-695c895bd6-lrbmf\" (UID: \"7a90ede4-bd7c-4b11-8144-1d87d18fe69e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"
Sep 07 16:19:56.456357 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.456275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4cz\" (UniqueName: \"kubernetes.io/projected/61dd7487-9e46-4d8b-aba1-7d208e5866fc-kube-api-access-bj4cz\") pod \"insights-operator-76bcd6ccf5-ncq5c\" (UID: \"61dd7487-9e46-4d8b-aba1-7d208e5866fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.543894 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.543790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.543894 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.543863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2aa09c03-bffd-4df4-99b1-3b80887591cc-nginx-conf\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.544081 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.543954    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:19:56.544081 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.544019    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.044003958 +0000 UTC m=+35.688299246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:19:56.544376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.544359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2aa09c03-bffd-4df4-99b1-3b80887591cc-nginx-conf\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:56.557251 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.557234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c"
Sep 07 16:19:56.565970 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.565952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:19:56.575127 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.575107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"
Sep 07 16:19:56.604996 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.604794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"
Sep 07 16:19:56.612663 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.612453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr97m"
Sep 07 16:19:56.620136 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.619952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh"
Sep 07 16:19:56.627031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.627007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"
Sep 07 16:19:56.847916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.847302    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bxjnn"]
Sep 07 16:19:56.850908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.850138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ncq5c"]
Sep 07 16:19:56.854032 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.854004    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dd7487_9e46_4d8b_aba1_7d208e5866fc.slice/crio-3662e6cc1c340f7272d3dbc28d42e2e8212ab44822b1b8c283d2e184b0cb1113 WatchSource:0}: Error finding container 3662e6cc1c340f7272d3dbc28d42e2e8212ab44822b1b8c283d2e184b0cb1113: Status 404 returned error can't find the container with id 3662e6cc1c340f7272d3dbc28d42e2e8212ab44822b1b8c283d2e184b0cb1113
Sep 07 16:19:56.861213 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.861188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf"]
Sep 07 16:19:56.864027 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.864006    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a90ede4_bd7c_4b11_8144_1d87d18fe69e.slice/crio-7fe5dadb9ee0977f042ffa4d6916a3fb8ede440bfb2873221f47dd35282b53b5 WatchSource:0}: Error finding container 7fe5dadb9ee0977f042ffa4d6916a3fb8ede440bfb2873221f47dd35282b53b5: Status 404 returned error can't find the container with id 7fe5dadb9ee0977f042ffa4d6916a3fb8ede440bfb2873221f47dd35282b53b5
Sep 07 16:19:56.882433 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.882406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:19:56.882736 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.882723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:19:56.882980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.882961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:19:56.885547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.885529    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 16:19:56.885731 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.885705    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 16:19:56.885848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.885817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9lqj4\""
Sep 07 16:19:56.885969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.885948    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4f7n\""
Sep 07 16:19:56.919069 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.919048    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg"]
Sep 07 16:19:56.922380 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.922358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc79fad_e548_465c_ab98_afaefef818d5.slice/crio-a5d849606766cdc7e6bfd8b44e1b7d0f09d0963c8e10f85187826ae346691283 WatchSource:0}: Error finding container a5d849606766cdc7e6bfd8b44e1b7d0f09d0963c8e10f85187826ae346691283: Status 404 returned error can't find the container with id a5d849606766cdc7e6bfd8b44e1b7d0f09d0963c8e10f85187826ae346691283
Sep 07 16:19:56.948002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.947978    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gqclh"]
Sep 07 16:19:56.948539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.948539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948628    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948701    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948730    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948765    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948781    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:56.948813 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948708    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.9486765 +0000 UTC m=+36.592971806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948847    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.948826333 +0000 UTC m=+36.593121627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.948901    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.948889316 +0000 UTC m=+36.593184604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.948974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.949109    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.949096513 +0000 UTC m=+36.593391829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.949133    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.949122784 +0000 UTC m=+36.593418134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.949161    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:19:56.949490 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:56.949208    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:57.949193107 +0000 UTC m=+36.593488409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:19:56.950825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.950800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr97m"]
Sep 07 16:19:56.951782 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:56.951711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw"]
Sep 07 16:19:56.952891 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.952866    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce3af34_7b9b_4b7d_a3a7_d182414b5330.slice/crio-c65773023e1c422adecea2b2ebe779ca2cf92e16668045e4ae26902597983845 WatchSource:0}: Error finding container c65773023e1c422adecea2b2ebe779ca2cf92e16668045e4ae26902597983845: Status 404 returned error can't find the container with id c65773023e1c422adecea2b2ebe779ca2cf92e16668045e4ae26902597983845
Sep 07 16:19:56.955075 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.955038    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fbd4161_cad8_431c_9de9_a21c2fcd6fff.slice/crio-ae064dd76ef0d828df9e0a54d56e4f443733355bcfcae21b6cec05a9bd56794e WatchSource:0}: Error finding container ae064dd76ef0d828df9e0a54d56e4f443733355bcfcae21b6cec05a9bd56794e: Status 404 returned error can't find the container with id ae064dd76ef0d828df9e0a54d56e4f443733355bcfcae21b6cec05a9bd56794e
Sep 07 16:19:56.955812 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:19:56.955792    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cbfe08_e923_4162_b279_2f1514acff96.slice/crio-e878bac3ef62af2297258d130b4f8e7005ca2d824cebe056cc710f7062301151 WatchSource:0}: Error finding container e878bac3ef62af2297258d130b4f8e7005ca2d824cebe056cc710f7062301151: Status 404 returned error can't find the container with id e878bac3ef62af2297258d130b4f8e7005ca2d824cebe056cc710f7062301151
Sep 07 16:19:57.050356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.050323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:57.050569 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.050485    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:19:57.050569 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.050556    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:19:58.050535804 +0000 UTC m=+36.694831093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:19:57.154173 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.154100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh" event={"ID":"fce3af34-7b9b-4b7d-a3a7-d182414b5330","Type":"ContainerStarted","Data":"c65773023e1c422adecea2b2ebe779ca2cf92e16668045e4ae26902597983845"}
Sep 07 16:19:57.155196 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.155172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg" event={"ID":"fdc79fad-e548-465c-ab98-afaefef818d5","Type":"ContainerStarted","Data":"a5d849606766cdc7e6bfd8b44e1b7d0f09d0963c8e10f85187826ae346691283"}
Sep 07 16:19:57.156198 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.156175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" event={"ID":"d3d62372-86a4-46e4-bb1e-9a4622096b13","Type":"ContainerStarted","Data":"60e3b5f04adede3fa72dfb5e0b42d7be98958b22a304d62f8ca5638723076c11"}
Sep 07 16:19:57.157210 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.157192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr97m" event={"ID":"a7cbfe08-e923-4162-b279-2f1514acff96","Type":"ContainerStarted","Data":"e878bac3ef62af2297258d130b4f8e7005ca2d824cebe056cc710f7062301151"}
Sep 07 16:19:57.158331 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.158310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c" event={"ID":"61dd7487-9e46-4d8b-aba1-7d208e5866fc","Type":"ContainerStarted","Data":"3662e6cc1c340f7272d3dbc28d42e2e8212ab44822b1b8c283d2e184b0cb1113"}
Sep 07 16:19:57.159403 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.159377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw" event={"ID":"9fbd4161-cad8-431c-9de9-a21c2fcd6fff","Type":"ContainerStarted","Data":"ae064dd76ef0d828df9e0a54d56e4f443733355bcfcae21b6cec05a9bd56794e"}
Sep 07 16:19:57.160684 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.160650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf" event={"ID":"7a90ede4-bd7c-4b11-8144-1d87d18fe69e","Type":"ContainerStarted","Data":"7fe5dadb9ee0977f042ffa4d6916a3fb8ede440bfb2873221f47dd35282b53b5"}
Sep 07 16:19:57.959041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959177    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959227    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.959213757 +0000 UTC m=+38.603509045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959177    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959276    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959334    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959348    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959335    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.95931703 +0000 UTC m=+38.603612332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959392    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.959381782 +0000 UTC m=+38.603677069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959403    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.959397552 +0000 UTC m=+38.603692841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:57.959461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959572    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.959562697 +0000 UTC m=+38.603857988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:19:57.959630 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959615    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:19:57.960227 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:57.959635    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:19:59.959629199 +0000 UTC m=+38.603924487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:19:58.060426 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:58.060381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:19:58.060584 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:58.060537    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:19:58.060637 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:58.060619    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:20:00.06060067 +0000 UTC m=+38.704895963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:19:59.179312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.178056    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="c7d6c7c80287d6506771ccfda4ad43660792e0bb922935d213dc89ccf55f0529" exitCode=0
Sep 07 16:19:59.179312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.178132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"c7d6c7c80287d6506771ccfda4ad43660792e0bb922935d213dc89ccf55f0529"}
Sep 07 16:19:59.984458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:19:59.984657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:59.984657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:19:59.984657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:19:59.984843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:19:59.984843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:19:59.984716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:19:59.984950 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.984893    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:19:59.984950 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.984910    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:19:59.985043 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.984969    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.984951116 +0000 UTC m=+42.629246407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:19:59.985043 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985040    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:19:59.985146 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985073    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.98506229 +0000 UTC m=+42.629357592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:19:59.985146 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985137    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.985126984 +0000 UTC m=+42.629422280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985189    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985236    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.985224763 +0000 UTC m=+42.629520067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985257    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985296    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.985284988 +0000 UTC m=+42.629580278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985363    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:19:59.985414 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:19:59.985392    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:03.985382957 +0000 UTC m=+42.629678246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:20:00.086273 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:00.086148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:00.086462 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:00.086333    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:20:00.086462 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:00.086391    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:20:04.08637708 +0000 UTC m=+42.730672369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:20:00.189376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:00.189335    2579 generic.go:356] "Generic (PLEG): container finished" podID="1edc37cf-dc0d-4e09-977f-30a393e7be84" containerID="40daada1c5a82dbe46e22a1c2fa492965c20b84165cd230a6d27f475216793f3" exitCode=0
Sep 07 16:20:00.189860 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:00.189433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerDied","Data":"40daada1c5a82dbe46e22a1c2fa492965c20b84165cd230a6d27f475216793f3"}
Sep 07 16:20:01.196671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:01.196583    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" event={"ID":"1edc37cf-dc0d-4e09-977f-30a393e7be84","Type":"ContainerStarted","Data":"bdc5c67facc605d277480beb74732292d30cff7395865c0e36d44b997614e386"}
Sep 07 16:20:01.218276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:01.218226    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2zdjd" podStartSLOduration=8.882288034 podStartE2EDuration="39.218203695s" totalImagesPullingTime="30.335915661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:01.216490645 +0000 UTC m=+39.860785976" watchObservedRunningTime="2026-09-07 16:20:01.218203695 +0000 UTC m=+39.862499007"
Sep 07 16:20:04.026432 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:04.026432 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026544    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026562    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026591    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026578105 +0000 UTC m=+50.670873393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026689    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026727    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026699343 +0000 UTC m=+50.670994633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026777    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026764325 +0000 UTC m=+50.671059613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026787    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026801    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026837    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026823935 +0000 UTC m=+50.671119224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:20:04.026920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.026874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:04.027324 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026958    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:20:04.027324 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.026972    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026964148 +0000 UTC m=+50.671259435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:20:04.027324 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.027000    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.026992804 +0000 UTC m=+50.671288091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:20:04.127694 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:04.127657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:04.127888 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.127864    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:20:04.127971 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:04.127957    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:20:12.127934256 +0000 UTC m=+50.772229546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:20:07.215209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.215167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh" event={"ID":"fce3af34-7b9b-4b7d-a3a7-d182414b5330","Type":"ContainerStarted","Data":"281ec71a96e6c912f2e71d76850e6e199e3441f414017c0627430e947f7a0fe6"}
Sep 07 16:20:07.217563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.217183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg" event={"ID":"fdc79fad-e548-465c-ab98-afaefef818d5","Type":"ContainerStarted","Data":"9b21f42cd87f3145846ce070bbc9d3956fc4c95dcc83f1d5605e90c9799ac9c5"}
Sep 07 16:20:07.219265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.219246    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/0.log"
Sep 07 16:20:07.219384 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.219282    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3d62372-86a4-46e4-bb1e-9a4622096b13" containerID="c9fca00f66729653e9b538f007728d672211255397a32f2009c40bbc5d4632fa" exitCode=255
Sep 07 16:20:07.219384 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.219357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" event={"ID":"d3d62372-86a4-46e4-bb1e-9a4622096b13","Type":"ContainerDied","Data":"c9fca00f66729653e9b538f007728d672211255397a32f2009c40bbc5d4632fa"}
Sep 07 16:20:07.219561 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.219544    2579 scope.go:117] "RemoveContainer" containerID="c9fca00f66729653e9b538f007728d672211255397a32f2009c40bbc5d4632fa"
Sep 07 16:20:07.222098 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.220944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr97m" event={"ID":"a7cbfe08-e923-4162-b279-2f1514acff96","Type":"ContainerStarted","Data":"f8bb61f6bbbedf8866b6249da7ec31616d42b9b300e77c4df77b4265cd8f7a63"}
Sep 07 16:20:07.222906 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.222857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c" event={"ID":"61dd7487-9e46-4d8b-aba1-7d208e5866fc","Type":"ContainerStarted","Data":"b30123ffe937b2dccabd7261b4045613a3a39464c32b5bbafa55808e3af48d48"}
Sep 07 16:20:07.224489 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.224459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw" event={"ID":"9fbd4161-cad8-431c-9de9-a21c2fcd6fff","Type":"ContainerStarted","Data":"a4c0c6fb005fe0fe2c614c5f64c8f4f4f26fe1f029826d83e218bde098b2f406"}
Sep 07 16:20:07.226026 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.225984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf" event={"ID":"7a90ede4-bd7c-4b11-8144-1d87d18fe69e","Type":"ContainerStarted","Data":"e79825a564490b02009e9889adce29d9e0c26030eb222d50d4c5bec09ed896db"}
Sep 07 16:20:07.232390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.232343    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-gqclh" podStartSLOduration=28.844299132 podStartE2EDuration="38.232330635s" totalImagesPullingTime="9.388031503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.231627522 +0000 UTC m=+45.875922833" watchObservedRunningTime="2026-09-07 16:20:07.232330635 +0000 UTC m=+45.876625939"
Sep 07 16:20:07.265660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.265617    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-lrbmf" podStartSLOduration=34.884893933 podStartE2EDuration="44.265603813s" totalImagesPullingTime="9.38070988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.246875051 +0000 UTC m=+45.891170376" watchObservedRunningTime="2026-09-07 16:20:07.265603813 +0000 UTC m=+45.909899117"
Sep 07 16:20:07.283637 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.283595    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg" podStartSLOduration=32.875644304 podStartE2EDuration="42.283578859s" totalImagesPullingTime="9.407934555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.26631751 +0000 UTC m=+45.910612824" watchObservedRunningTime="2026-09-07 16:20:07.283578859 +0000 UTC m=+45.927874168"
Sep 07 16:20:07.297898 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.297844    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw" podStartSLOduration=30.919818365 podStartE2EDuration="40.297823108s" totalImagesPullingTime="9.378004743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.297029956 +0000 UTC m=+45.941325259" watchObservedRunningTime="2026-09-07 16:20:07.297823108 +0000 UTC m=+45.942118420"
Sep 07 16:20:07.314823 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.314758    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c" podStartSLOduration=34.835247697 podStartE2EDuration="44.314718846s" totalImagesPullingTime="9.479471149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.313708973 +0000 UTC m=+45.958004302" watchObservedRunningTime="2026-09-07 16:20:07.314718846 +0000 UTC m=+45.959014158"
Sep 07 16:20:07.328043 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.327986    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jr97m" podStartSLOduration=1.951957274 podStartE2EDuration="11.327970458s" totalImagesPullingTime="9.376013184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:07.327960769 +0000 UTC m=+45.972256093" watchObservedRunningTime="2026-09-07 16:20:07.327970458 +0000 UTC m=+45.972265768"
Sep 07 16:20:07.930047 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.929999    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mj6rz"]
Sep 07 16:20:07.953954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.953928    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mj6rz"]
Sep 07 16:20:07.954066 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.954045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:07.956023 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.955998    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 16:20:07.956162 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.956060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 16:20:07.956162 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:07.956079    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s7rhg\""
Sep 07 16:20:08.063924 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.063879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttsmt\" (UniqueName: \"kubernetes.io/projected/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-api-access-ttsmt\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.064123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.063948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-data-volume\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.064123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.063989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.064123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.064066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-crio-socket\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.064285 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.064149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.164803 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.164753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttsmt\" (UniqueName: \"kubernetes.io/projected/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-api-access-ttsmt\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.164832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-data-volume\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.164859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.164894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-crio-socket\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.164984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165197 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.165030    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:08.165197 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.165106    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:08.665085609 +0000 UTC m=+47.309380910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:08.165197 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.165132    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 16:20:08.165197 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.165185    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:08.665168332 +0000 UTC m=+47.309463640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : configmap "kube-rbac-proxy" not found
Sep 07 16:20:08.165339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.165192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-crio-socket\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.165339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.165272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-data-volume\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.174112 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.174087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttsmt\" (UniqueName: \"kubernetes.io/projected/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-api-access-ttsmt\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.231556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.231528    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:20:08.232087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.232066    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/0.log"
Sep 07 16:20:08.232139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.232114    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3d62372-86a4-46e4-bb1e-9a4622096b13" containerID="e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0" exitCode=255
Sep 07 16:20:08.232249 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.232223    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" event={"ID":"d3d62372-86a4-46e4-bb1e-9a4622096b13","Type":"ContainerDied","Data":"e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0"}
Sep 07 16:20:08.232332 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.232317    2579 scope.go:117] "RemoveContainer" containerID="c9fca00f66729653e9b538f007728d672211255397a32f2009c40bbc5d4632fa"
Sep 07 16:20:08.232667 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.232448    2579 scope.go:117] "RemoveContainer" containerID="e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0"
Sep 07 16:20:08.232667 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.232647    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bxjnn_openshift-console-operator(d3d62372-86a4-46e4-bb1e-9a4622096b13)\"" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" podUID="d3d62372-86a4-46e4-bb1e-9a4622096b13"
Sep 07 16:20:08.670661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.670583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.670838 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:08.670679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:08.670838 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.670769    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 16:20:08.670838 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.670768    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:08.670838 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.670838    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:09.670824915 +0000 UTC m=+48.315120203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : configmap "kube-rbac-proxy" not found
Sep 07 16:20:08.670977 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:08.670851    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:09.670845556 +0000 UTC m=+48.315140844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:09.237515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:09.237486    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:20:09.237916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:09.237830    2579 scope.go:117] "RemoveContainer" containerID="e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0"
Sep 07 16:20:09.237994 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:09.237975    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bxjnn_openshift-console-operator(d3d62372-86a4-46e4-bb1e-9a4622096b13)\"" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" podUID="d3d62372-86a4-46e4-bb1e-9a4622096b13"
Sep 07 16:20:09.680453 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:09.680355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:09.680453 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:09.680448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:09.680644 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:09.680505    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:09.680644 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:09.680570    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 16:20:09.680644 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:09.680582    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:11.68056615 +0000 UTC m=+50.324861438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:09.680644 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:09.680617    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:11.680604637 +0000 UTC m=+50.324899925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : configmap "kube-rbac-proxy" not found
Sep 07 16:20:09.801924 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:09.801896    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zhl2d_ec32b97f-faca-4bcb-8733-9b95a727a079/dns-node-resolver/0.log"
Sep 07 16:20:10.801513 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:10.801485    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xcftb_c1746538-b2dc-4bf1-be5e-1bb8fa5bde10/node-ca/0.log"
Sep 07 16:20:11.201888 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:11.201837    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr97m_a7cbfe08-e923-4162-b279-2f1514acff96/serve-healthcheck-canary/0.log"
Sep 07 16:20:11.700163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:11.700133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:11.700325 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:11.700229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:11.700325 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:11.700296    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:11.700437 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:11.700361    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:15.700345176 +0000 UTC m=+54.344640463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:11.700437 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:11.700365    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 16:20:11.700437 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:11.700428    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:15.70041098 +0000 UTC m=+54.344706281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : configmap "kube-rbac-proxy" not found
Sep 07 16:20:12.104718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.104632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:20:12.104718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.104673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.104810    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.104829    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.104815031 +0000 UTC m=+66.749110319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : configmap references non-existent config key: service-ca.crt
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.104866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls podName:fcd813e3-85db-47fc-8d80-d172471725c9 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.104852576 +0000 UTC m=+66.749147868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9jb46" (UID: "fcd813e3-85db-47fc-8d80-d172471725c9") : secret "samples-operator-tls" not found
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.104922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.104949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.104995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105066    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105095    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105104    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls podName:de9dc804-3b56-4941-8d7f-dae66f7ffd14 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.105091666 +0000 UTC m=+66.749386969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7whbp" (UID: "de9dc804-3b56-4941-8d7f-dae66f7ffd14") : secret "cluster-monitoring-operator-tls" not found
Sep 07 16:20:12.105126 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105095    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.105138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105157    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls podName:6adcc7c9-5071-43f7-8179-31900ce7f8c2 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.105132225 +0000 UTC m=+66.749427518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls") pod "dns-default-r9h7m" (UID: "6adcc7c9-5071-43f7-8179-31900ce7f8c2") : secret "dns-default-metrics-tls" not found
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105197    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105201    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs podName:d86f2ea3-df9d-4ce5-af14-0e49782f295a nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.105172318 +0000 UTC m=+66.749467610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs") pod "router-default-8688c84cc4-cltmt" (UID: "d86f2ea3-df9d-4ce5-af14-0e49782f295a") : secret "router-metrics-certs-default" not found
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105207    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89958778-plt8b: secret "image-registry-tls" not found
Sep 07 16:20:12.105411 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.105238    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls podName:8d9c010b-ba99-4b10-8efd-195803e44aa6 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.105230942 +0000 UTC m=+66.749526229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls") pod "image-registry-89958778-plt8b" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6") : secret "image-registry-tls" not found
Sep 07 16:20:12.205671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:12.205636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:12.205866 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.205809    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 16:20:12.205909 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:12.205871    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert podName:2aa09c03-bffd-4df4-99b1-3b80887591cc nodeName:}" failed. No retries permitted until 2026-09-07 16:20:28.20585662 +0000 UTC m=+66.850151907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qc4gc" (UID: "2aa09c03-bffd-4df4-99b1-3b80887591cc") : secret "networking-console-plugin-cert" not found
Sep 07 16:20:15.733729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:15.733675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:15.734181 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:15.733823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:15.734181 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:15.733965    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:15.734181 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:15.734027    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls podName:5d25d52d-ad0c-4ff0-bdea-3da2e656caaa nodeName:}" failed. No retries permitted until 2026-09-07 16:20:23.734014063 +0000 UTC m=+62.378309350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj6rz" (UID: "5d25d52d-ad0c-4ff0-bdea-3da2e656caaa") : secret "insights-runtime-extractor-tls" not found
Sep 07 16:20:15.734314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:15.734294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:16.566809 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:16.566778    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:20:16.566986 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:16.566859    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:20:16.567185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:16.567173    2579 scope.go:117] "RemoveContainer" containerID="e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0"
Sep 07 16:20:16.567365 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:16.567347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bxjnn_openshift-console-operator(d3d62372-86a4-46e4-bb1e-9a4622096b13)\"" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" podUID="d3d62372-86a4-46e4-bb1e-9a4622096b13"
Sep 07 16:20:22.151146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:22.151113    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsjqp"
Sep 07 16:20:23.807216 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:23.807168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:23.809682 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:23.809661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d25d52d-ad0c-4ff0-bdea-3da2e656caaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj6rz\" (UID: \"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa\") " pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:23.867916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:23.867884    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s7rhg\""
Sep 07 16:20:23.876687 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:23.876662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mj6rz"
Sep 07 16:20:24.150472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:24.150396    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mj6rz"]
Sep 07 16:20:24.153112 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:24.153089    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d25d52d_ad0c_4ff0_bdea_3da2e656caaa.slice/crio-7afabc014bc406d3da1421e69dae947df9a056197dd54ddda6f525d430efd98d WatchSource:0}: Error finding container 7afabc014bc406d3da1421e69dae947df9a056197dd54ddda6f525d430efd98d: Status 404 returned error can't find the container with id 7afabc014bc406d3da1421e69dae947df9a056197dd54ddda6f525d430efd98d
Sep 07 16:20:24.288033 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:24.287999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj6rz" event={"ID":"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa","Type":"ContainerStarted","Data":"3b133b40d7c88c023376f85e6d84cab9c95bd3ce90ce305d1d099578af5bdcf0"}
Sep 07 16:20:24.288170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:24.288058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj6rz" event={"ID":"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa","Type":"ContainerStarted","Data":"7afabc014bc406d3da1421e69dae947df9a056197dd54ddda6f525d430efd98d"}
Sep 07 16:20:25.292906 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:25.292822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj6rz" event={"ID":"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa","Type":"ContainerStarted","Data":"8211570cf2a8b9f3898c39f31f45f3a1afc3f3681d3d336d998ab6cd549d36af"}
Sep 07 16:20:27.302085 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.302047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj6rz" event={"ID":"5d25d52d-ad0c-4ff0-bdea-3da2e656caaa","Type":"ContainerStarted","Data":"9041f4870dd6c435b9870760fb842b468957aad7f175bce42487993644857dc9"}
Sep 07 16:20:27.317385 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.317328    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mj6rz" podStartSLOduration=18.091698624 podStartE2EDuration="20.317310816s" totalImagesPullingTime="2.225612192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:20:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:27.316388082 +0000 UTC m=+65.960683413" watchObservedRunningTime="2026-09-07 16:20:27.317310816 +0000 UTC m=+65.961606125"
Sep 07 16:20:27.644861 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.644767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:20:27.644861 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.644811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:20:27.646925 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.646909    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 16:20:27.646990 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.646971    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 16:20:27.657931 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.657892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60c3d654-54ef-47cd-bc10-3f1173214ca8-original-pull-secret\") pod \"global-pull-secret-syncer-89lj5\" (UID: \"60c3d654-54ef-47cd-bc10-3f1173214ca8\") " pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:20:27.658113 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.658092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81a54d9-b6b8-4822-b998-e823d5ba34f0-metrics-certs\") pod \"network-metrics-daemon-zj7m6\" (UID: \"d81a54d9-b6b8-4822-b998-e823d5ba34f0\") " pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:20:27.746167 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.746112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:20:27.748907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.748872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6kws\" (UniqueName: \"kubernetes.io/projected/6456325c-0cf2-4af9-b979-d3cd149e8014-kube-api-access-t6kws\") pod \"network-check-target-pc54l\" (UID: \"6456325c-0cf2-4af9-b979-d3cd149e8014\") " pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:20:27.793591 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.793536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89lj5"
Sep 07 16:20:27.800868 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.800850    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4f7n\""
Sep 07 16:20:27.806034 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.806008    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9lqj4\""
Sep 07 16:20:27.809548 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.809524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zj7m6"
Sep 07 16:20:27.814854 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:27.814823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:20:28.068263 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.068227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89lj5"]
Sep 07 16:20:28.071361 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.071332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c3d654_54ef_47cd_bc10_3f1173214ca8.slice/crio-f303d142cedcb9ecd5c32bddb529c3dcbc107808a29bf2190b7e1cd8d6728971 WatchSource:0}: Error finding container f303d142cedcb9ecd5c32bddb529c3dcbc107808a29bf2190b7e1cd8d6728971: Status 404 returned error can't find the container with id f303d142cedcb9ecd5c32bddb529c3dcbc107808a29bf2190b7e1cd8d6728971
Sep 07 16:20:28.104703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.104672    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zj7m6"]
Sep 07 16:20:28.107573 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.107544    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81a54d9_b6b8_4822_b998_e823d5ba34f0.slice/crio-669797126f95d4202b39b9c20420e1e8f70119ec0a528035ad6fc03f5297c5ff WatchSource:0}: Error finding container 669797126f95d4202b39b9c20420e1e8f70119ec0a528035ad6fc03f5297c5ff: Status 404 returned error can't find the container with id 669797126f95d4202b39b9c20420e1e8f70119ec0a528035ad6fc03f5297c5ff
Sep 07 16:20:28.110132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.110108    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pc54l"]
Sep 07 16:20:28.113314 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.113289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6456325c_0cf2_4af9_b979_d3cd149e8014.slice/crio-eaf47765199ec0762f55c9a753e14917e3183e2bc9dbd3ac75f14fea9147db05 WatchSource:0}: Error finding container eaf47765199ec0762f55c9a753e14917e3183e2bc9dbd3ac75f14fea9147db05: Status 404 returned error can't find the container with id eaf47765199ec0762f55c9a753e14917e3183e2bc9dbd3ac75f14fea9147db05
Sep 07 16:20:28.149838 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.149802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:20:28.150001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.149858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:28.150001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.149912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:20:28.150001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.149938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:28.150001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.149999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:28.150354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.150053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:28.151006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.150941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d86f2ea3-df9d-4ce5-af14-0e49782f295a-service-ca-bundle\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:28.152758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.152691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/de9dc804-3b56-4941-8d7f-dae66f7ffd14-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7whbp\" (UID: \"de9dc804-3b56-4941-8d7f-dae66f7ffd14\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:20:28.152874 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.152825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d86f2ea3-df9d-4ce5-af14-0e49782f295a-metrics-certs\") pod \"router-default-8688c84cc4-cltmt\" (UID: \"d86f2ea3-df9d-4ce5-af14-0e49782f295a\") " pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:28.152874 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.152861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"image-registry-89958778-plt8b\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") " pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:28.153216 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.153199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6adcc7c9-5071-43f7-8179-31900ce7f8c2-metrics-tls\") pod \"dns-default-r9h7m\" (UID: \"6adcc7c9-5071-43f7-8179-31900ce7f8c2\") " pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:28.153333 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.153315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcd813e3-85db-47fc-8d80-d172471725c9-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9jb46\" (UID: \"fcd813e3-85db-47fc-8d80-d172471725c9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:20:28.251157 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.251120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:28.253687 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.253665    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2aa09c03-bffd-4df4-99b1-3b80887591cc-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qc4gc\" (UID: \"2aa09c03-bffd-4df4-99b1-3b80887591cc\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:28.306036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.305998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89lj5" event={"ID":"60c3d654-54ef-47cd-bc10-3f1173214ca8","Type":"ContainerStarted","Data":"f303d142cedcb9ecd5c32bddb529c3dcbc107808a29bf2190b7e1cd8d6728971"}
Sep 07 16:20:28.307608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.307574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pc54l" event={"ID":"6456325c-0cf2-4af9-b979-d3cd149e8014","Type":"ContainerStarted","Data":"f07008c711c4aeb39736f984fb35669465f631939c66b5c2520d3e73f9828ff3"}
Sep 07 16:20:28.307763 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.307614    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pc54l" event={"ID":"6456325c-0cf2-4af9-b979-d3cd149e8014","Type":"ContainerStarted","Data":"eaf47765199ec0762f55c9a753e14917e3183e2bc9dbd3ac75f14fea9147db05"}
Sep 07 16:20:28.307763 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.307646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:20:28.308801 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.308778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zj7m6" event={"ID":"d81a54d9-b6b8-4822-b998-e823d5ba34f0","Type":"ContainerStarted","Data":"669797126f95d4202b39b9c20420e1e8f70119ec0a528035ad6fc03f5297c5ff"}
Sep 07 16:20:28.321774 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.321653    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pc54l" podStartSLOduration=66.321634037 podStartE2EDuration="1m6.321634037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:28.320596542 +0000 UTC m=+66.964891854" watchObservedRunningTime="2026-09-07 16:20:28.321634037 +0000 UTC m=+66.965929363"
Sep 07 16:20:28.330584 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.330558    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pg7bg\""
Sep 07 16:20:28.339251 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.339227    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"
Sep 07 16:20:28.342275 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.342242    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qt7fz\""
Sep 07 16:20:28.350827 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.350806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:28.383565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.383529    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rdllp\""
Sep 07 16:20:28.391487 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.391461    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t2gln\""
Sep 07 16:20:28.391804 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.391732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:28.399708 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.399672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"
Sep 07 16:20:28.399950 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.399927    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c2wmp\""
Sep 07 16:20:28.408406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.408377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:28.438179 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.438137    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7chqs\""
Sep 07 16:20:28.446908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.446840    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc"
Sep 07 16:20:28.612214 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.612136    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp"]
Sep 07 16:20:28.615362 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.615331    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9dc804_3b56_4941_8d7f_dae66f7ffd14.slice/crio-ebdd6844a0cb4c1d5a9494dbbed7513e68e8444c3f460dfffd938fbb8d4ea5fa WatchSource:0}: Error finding container ebdd6844a0cb4c1d5a9494dbbed7513e68e8444c3f460dfffd938fbb8d4ea5fa: Status 404 returned error can't find the container with id ebdd6844a0cb4c1d5a9494dbbed7513e68e8444c3f460dfffd938fbb8d4ea5fa
Sep 07 16:20:28.638787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.638734    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:20:28.641529 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.641497    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9c010b_ba99_4b10_8efd_195803e44aa6.slice/crio-ad5563de4166e1dbaf439545f39c8b4df0a524e51558ebb346c77c582004e65a WatchSource:0}: Error finding container ad5563de4166e1dbaf439545f39c8b4df0a524e51558ebb346c77c582004e65a: Status 404 returned error can't find the container with id ad5563de4166e1dbaf439545f39c8b4df0a524e51558ebb346c77c582004e65a
Sep 07 16:20:28.696312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.696270    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46"]
Sep 07 16:20:28.699937 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.699853    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8688c84cc4-cltmt"]
Sep 07 16:20:28.706101 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.706061    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86f2ea3_df9d_4ce5_af14_0e49782f295a.slice/crio-00b2dda0ca58b4a107f1bfb68876a6a887d0822a53a4186ae96d98f64719f17e WatchSource:0}: Error finding container 00b2dda0ca58b4a107f1bfb68876a6a887d0822a53a4186ae96d98f64719f17e: Status 404 returned error can't find the container with id 00b2dda0ca58b4a107f1bfb68876a6a887d0822a53a4186ae96d98f64719f17e
Sep 07 16:20:28.720807 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.720198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9h7m"]
Sep 07 16:20:28.731010 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.730978    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adcc7c9_5071_43f7_8179_31900ce7f8c2.slice/crio-f5bf6355b13f556aabd62b13c0b0c0af2b9adb6e327ef004a6d0b172c9205e69 WatchSource:0}: Error finding container f5bf6355b13f556aabd62b13c0b0c0af2b9adb6e327ef004a6d0b172c9205e69: Status 404 returned error can't find the container with id f5bf6355b13f556aabd62b13c0b0c0af2b9adb6e327ef004a6d0b172c9205e69
Sep 07 16:20:28.735792 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.735768    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qc4gc"]
Sep 07 16:20:28.740100 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:28.740067    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa09c03_bffd_4df4_99b1_3b80887591cc.slice/crio-480992fa9b406898513c938a3e301a5abd68cd185a1ff78fcab524405006b935 WatchSource:0}: Error finding container 480992fa9b406898513c938a3e301a5abd68cd185a1ff78fcab524405006b935: Status 404 returned error can't find the container with id 480992fa9b406898513c938a3e301a5abd68cd185a1ff78fcab524405006b935
Sep 07 16:20:28.882829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:28.882730    2579 scope.go:117] "RemoveContainer" containerID="e62078b1adc24dc8caa9ba1afaee45d146e6df6c0e72d32c289c18c5ac76eef0"
Sep 07 16:20:29.317616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.317586    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:20:29.318605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.317701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" event={"ID":"d3d62372-86a4-46e4-bb1e-9a4622096b13","Type":"ContainerStarted","Data":"dff72f5ee6f8e6340ed237f6b6c654549b927cd0e06a8dd3ddc77080c50477a0"}
Sep 07 16:20:29.318605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.318029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:20:29.319529 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.319498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46" event={"ID":"fcd813e3-85db-47fc-8d80-d172471725c9","Type":"ContainerStarted","Data":"4217b7e44588bca35439e6d2ac613fe839083cbd2d705e2907a725e48a0b2876"}
Sep 07 16:20:29.321271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.321247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc" event={"ID":"2aa09c03-bffd-4df4-99b1-3b80887591cc","Type":"ContainerStarted","Data":"480992fa9b406898513c938a3e301a5abd68cd185a1ff78fcab524405006b935"}
Sep 07 16:20:29.323398 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.323370    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89958778-plt8b" event={"ID":"8d9c010b-ba99-4b10-8efd-195803e44aa6","Type":"ContainerStarted","Data":"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"}
Sep 07 16:20:29.323520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.323418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89958778-plt8b" event={"ID":"8d9c010b-ba99-4b10-8efd-195803e44aa6","Type":"ContainerStarted","Data":"ad5563de4166e1dbaf439545f39c8b4df0a524e51558ebb346c77c582004e65a"}
Sep 07 16:20:29.323685 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.323645    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:29.325137 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.325107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp" event={"ID":"de9dc804-3b56-4941-8d7f-dae66f7ffd14","Type":"ContainerStarted","Data":"ebdd6844a0cb4c1d5a9494dbbed7513e68e8444c3f460dfffd938fbb8d4ea5fa"}
Sep 07 16:20:29.326802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.326776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8688c84cc4-cltmt" event={"ID":"d86f2ea3-df9d-4ce5-af14-0e49782f295a","Type":"ContainerStarted","Data":"e513ab3f4d01c2da3c7dfa7715a4e8457f473f052fbb936ad384028b99a84b13"}
Sep 07 16:20:29.326912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.326821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8688c84cc4-cltmt" event={"ID":"d86f2ea3-df9d-4ce5-af14-0e49782f295a","Type":"ContainerStarted","Data":"00b2dda0ca58b4a107f1bfb68876a6a887d0822a53a4186ae96d98f64719f17e"}
Sep 07 16:20:29.328665 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.328633    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9h7m" event={"ID":"6adcc7c9-5071-43f7-8179-31900ce7f8c2","Type":"ContainerStarted","Data":"f5bf6355b13f556aabd62b13c0b0c0af2b9adb6e327ef004a6d0b172c9205e69"}
Sep 07 16:20:29.349799 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.349696    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-8688c84cc4-cltmt" podStartSLOduration=66.349672564 podStartE2EDuration="1m6.349672564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:29.349559859 +0000 UTC m=+67.993855169" watchObservedRunningTime="2026-09-07 16:20:29.349672564 +0000 UTC m=+67.993967875"
Sep 07 16:20:29.350583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.350545    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-bxjnn" podStartSLOduration=56.868368766 podStartE2EDuration="1m6.3505335s" totalImagesPullingTime="9.482164734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:29.333147422 +0000 UTC m=+67.977442733" watchObservedRunningTime="2026-09-07 16:20:29.3505335 +0000 UTC m=+67.994828809"
Sep 07 16:20:29.367130 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.367086    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-89958778-plt8b" podStartSLOduration=67.367069137 podStartE2EDuration="1m7.367069137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:29.36592533 +0000 UTC m=+68.010220663" watchObservedRunningTime="2026-09-07 16:20:29.367069137 +0000 UTC m=+68.011364444"
Sep 07 16:20:29.393862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.393829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:29.395685 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:29.395656    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:30.039821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.039786    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-bxjnn"
Sep 07 16:20:30.343856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.341483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zj7m6" event={"ID":"d81a54d9-b6b8-4822-b998-e823d5ba34f0","Type":"ContainerStarted","Data":"b38cd3b4546437a14341ae09e267eba5576e16b35bbd564c25ae9703ce353f77"}
Sep 07 16:20:30.343856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.341545    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zj7m6" event={"ID":"d81a54d9-b6b8-4822-b998-e823d5ba34f0","Type":"ContainerStarted","Data":"7de96a37b66d174ec3d3c656744a38b04e7f465c37dabfdcbbd2f3b97a9f7146"}
Sep 07 16:20:30.343856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.342875    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:30.344704 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.344643    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8688c84cc4-cltmt"
Sep 07 16:20:30.359942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:30.359881    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zj7m6" podStartSLOduration=67.10853298 podStartE2EDuration="1m8.359853951s" totalImagesPullingTime="1.251320971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:30.359239004 +0000 UTC m=+69.003534313" watchObservedRunningTime="2026-09-07 16:20:30.359853951 +0000 UTC m=+69.004149262"
Sep 07 16:20:34.262149 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.262122    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"]
Sep 07 16:20:34.292053 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.292031    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"]
Sep 07 16:20:34.292147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.292123    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.297017 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.296993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 07 16:20:34.297222 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.297208    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 07 16:20:34.297932 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.297915    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 07 16:20:34.298088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.298070    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 07 16:20:34.302853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.302664    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-78hvd\""
Sep 07 16:20:34.369417 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.369382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc" event={"ID":"2aa09c03-bffd-4df4-99b1-3b80887591cc","Type":"ContainerStarted","Data":"885acee05d403db60fd691aa75989a353580144d797258a39b6936be365b8326"}
Sep 07 16:20:34.371051 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.371024    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-zmd8z"]
Sep 07 16:20:34.403892 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.403723    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zmd8z"]
Sep 07 16:20:34.403892 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.403852    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:34.406100 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.406075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 07 16:20:34.406100 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.406091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-t9gzz\""
Sep 07 16:20:34.406301 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.406075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 07 16:20:34.408510 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.408488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32e94ad7-a852-4ee2-a4d5-81df3000856e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.408723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.408544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwjb9\" (UniqueName: \"kubernetes.io/projected/32e94ad7-a852-4ee2-a4d5-81df3000856e-kube-api-access-dwjb9\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.510800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.509598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwjb9\" (UniqueName: \"kubernetes.io/projected/32e94ad7-a852-4ee2-a4d5-81df3000856e-kube-api-access-dwjb9\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.510800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.509728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqtp\" (UniqueName: \"kubernetes.io/projected/c5c51f07-797b-4b83-b922-d930a9832775-kube-api-access-zmqtp\") pod \"downloads-85d98c998f-zmd8z\" (UID: \"c5c51f07-797b-4b83-b922-d930a9832775\") " pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:34.510800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.509783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32e94ad7-a852-4ee2-a4d5-81df3000856e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.513002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.512974    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32e94ad7-a852-4ee2-a4d5-81df3000856e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.527438 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.526259    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwjb9\" (UniqueName: \"kubernetes.io/projected/32e94ad7-a852-4ee2-a4d5-81df3000856e-kube-api-access-dwjb9\") pod \"managed-serviceaccount-addon-agent-56d5f67678-9mrz2\" (UID: \"32e94ad7-a852-4ee2-a4d5-81df3000856e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.611183 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.611151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqtp\" (UniqueName: \"kubernetes.io/projected/c5c51f07-797b-4b83-b922-d930a9832775-kube-api-access-zmqtp\") pod \"downloads-85d98c998f-zmd8z\" (UID: \"c5c51f07-797b-4b83-b922-d930a9832775\") " pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:34.623345 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.623315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqtp\" (UniqueName: \"kubernetes.io/projected/c5c51f07-797b-4b83-b922-d930a9832775-kube-api-access-zmqtp\") pod \"downloads-85d98c998f-zmd8z\" (UID: \"c5c51f07-797b-4b83-b922-d930a9832775\") " pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:34.637155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.637129    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"
Sep 07 16:20:34.724775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.724522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:34.898802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.898733    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"]
Sep 07 16:20:34.922119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.922077    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"]
Sep 07 16:20:34.922258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.922212    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:34.924443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.924422    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 07 16:20:34.924590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.924504    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sncrx\""
Sep 07 16:20:34.955436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:34.955406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2"]
Sep 07 16:20:34.958798 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:34.958774    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32e94ad7_a852_4ee2_a4d5_81df3000856e.slice/crio-42c9dabcf3719fd5c1e0c23240f0a5135cd4ed935cb325b46277b866d789e654 WatchSource:0}: Error finding container 42c9dabcf3719fd5c1e0c23240f0a5135cd4ed935cb325b46277b866d789e654: Status 404 returned error can't find the container with id 42c9dabcf3719fd5c1e0c23240f0a5135cd4ed935cb325b46277b866d789e654
Sep 07 16:20:35.014605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.014556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-thsnm\" (UID: \"9fcc16fa-d3b8-4fee-a71c-7604f892b610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:35.023288 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.023255    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zmd8z"]
Sep 07 16:20:35.026490 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:35.026463    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c51f07_797b_4b83_b922_d930a9832775.slice/crio-57dc6c743daf7567a157977cf4b64d3bfe7356fcf31138cc8efcc09d06d76df2 WatchSource:0}: Error finding container 57dc6c743daf7567a157977cf4b64d3bfe7356fcf31138cc8efcc09d06d76df2: Status 404 returned error can't find the container with id 57dc6c743daf7567a157977cf4b64d3bfe7356fcf31138cc8efcc09d06d76df2
Sep 07 16:20:35.114978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.114944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-thsnm\" (UID: \"9fcc16fa-d3b8-4fee-a71c-7604f892b610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:35.115142 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:35.115125    2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 16:20:35.115220 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:35.115207    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates podName:9fcc16fa-d3b8-4fee-a71c-7604f892b610 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:35.615184815 +0000 UTC m=+74.259480118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-thsnm" (UID: "9fcc16fa-d3b8-4fee-a71c-7604f892b610") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 16:20:35.377851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.377730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp" event={"ID":"de9dc804-3b56-4941-8d7f-dae66f7ffd14","Type":"ContainerStarted","Data":"73fd1bfa524bde996d9709be900209d7d55dce82aff5332ab07e74805ec9553e"}
Sep 07 16:20:35.379732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.379707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9h7m" event={"ID":"6adcc7c9-5071-43f7-8179-31900ce7f8c2","Type":"ContainerStarted","Data":"30851735aabff79953523fccb13e5a8a6ed09ec489ebc30da5c395e679e79389"}
Sep 07 16:20:35.379879 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.379769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9h7m" event={"ID":"6adcc7c9-5071-43f7-8179-31900ce7f8c2","Type":"ContainerStarted","Data":"5c79e482a82c5d27f05e4d23ef2f496800994c2253e554f58474366a3e0ede85"}
Sep 07 16:20:35.379879 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.379799    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:35.381424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.381404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89lj5" event={"ID":"60c3d654-54ef-47cd-bc10-3f1173214ca8","Type":"ContainerStarted","Data":"f5be154574bf6c5f0ae092a8bf42be91aeb38256e809bc138a54a3d65bec58df"}
Sep 07 16:20:35.383560 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.383531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46" event={"ID":"fcd813e3-85db-47fc-8d80-d172471725c9","Type":"ContainerStarted","Data":"253a04bc003fed8ac825c9193b12281dbb5d2173a01aec0600178d21ec4ffd10"}
Sep 07 16:20:35.383659 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.383578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46" event={"ID":"fcd813e3-85db-47fc-8d80-d172471725c9","Type":"ContainerStarted","Data":"2a9123dffc4c6875d783f9a79b2f05c9763dc196006be4730d3f1ae7aaec35d5"}
Sep 07 16:20:35.384908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.384877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zmd8z" event={"ID":"c5c51f07-797b-4b83-b922-d930a9832775","Type":"ContainerStarted","Data":"57dc6c743daf7567a157977cf4b64d3bfe7356fcf31138cc8efcc09d06d76df2"}
Sep 07 16:20:35.386095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.386077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2" event={"ID":"32e94ad7-a852-4ee2-a4d5-81df3000856e","Type":"ContainerStarted","Data":"42c9dabcf3719fd5c1e0c23240f0a5135cd4ed935cb325b46277b866d789e654"}
Sep 07 16:20:35.393251 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.393200    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7whbp" podStartSLOduration=66.775573087 podStartE2EDuration="1m12.393182681s" totalImagesPullingTime="5.617609594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:35.392151314 +0000 UTC m=+74.036446640" watchObservedRunningTime="2026-09-07 16:20:35.393182681 +0000 UTC m=+74.037477988"
Sep 07 16:20:35.406729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.406685    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-89lj5" podStartSLOduration=67.232220597 podStartE2EDuration="1m13.406665969s" totalImagesPullingTime="6.174445372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:35.406291835 +0000 UTC m=+74.050587161" watchObservedRunningTime="2026-09-07 16:20:35.406665969 +0000 UTC m=+74.050961280"
Sep 07 16:20:35.420972 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.420927    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-qc4gc" podStartSLOduration=33.978995858 podStartE2EDuration="39.42090868s" totalImagesPullingTime="5.441912822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:35.420202573 +0000 UTC m=+74.064497909" watchObservedRunningTime="2026-09-07 16:20:35.42090868 +0000 UTC m=+74.065203989"
Sep 07 16:20:35.435568 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.435519    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9jb46" podStartSLOduration=65.987267628 podStartE2EDuration="1m11.435505629s" totalImagesPullingTime="5.448238001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:35.434680526 +0000 UTC m=+74.078975862" watchObservedRunningTime="2026-09-07 16:20:35.435505629 +0000 UTC m=+74.079800939"
Sep 07 16:20:35.448827 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.448793    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:20:35.453673 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.453548    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r9h7m" podStartSLOduration=33.951932692 podStartE2EDuration="39.453533666s" totalImagesPullingTime="5.501600974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:35.453491306 +0000 UTC m=+74.097786665" watchObservedRunningTime="2026-09-07 16:20:35.453533666 +0000 UTC m=+74.097828976"
Sep 07 16:20:35.472986 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.472959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.473161 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.472957    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:20:35.475133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475106    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 07 16:20:35.475133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j7l4m\""
Sep 07 16:20:35.475338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475165    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 07 16:20:35.475338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475167    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 07 16:20:35.475338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 07 16:20:35.475338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.475108    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 07 16:20:35.619567 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-thsnm\" (UID: \"9fcc16fa-d3b8-4fee-a71c-7604f892b610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:35.619732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.619732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619635    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.619732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44brb\" (UniqueName: \"kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.619732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.619912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.619912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.619809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.620004 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:35.619946    2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 16:20:35.620004 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:20:35.619992    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates podName:9fcc16fa-d3b8-4fee-a71c-7604f892b610 nodeName:}" failed. No retries permitted until 2026-09-07 16:20:36.619978154 +0000 UTC m=+75.264273442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-thsnm" (UID: "9fcc16fa-d3b8-4fee-a71c-7604f892b610") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 16:20:35.720915 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.720862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721115 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.720921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44brb\" (UniqueName: \"kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721115 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.720959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721115 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.720983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.721319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721476 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.721408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721780 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.721726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.721886 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.721726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.722974 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.722951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.724227 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.724203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.725193 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.725150    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.728819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.728758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44brb\" (UniqueName: \"kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb\") pod \"console-7596864bf7-qnb8g\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") " pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:35.783501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:35.783447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:36.099333 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:36.099299    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:20:36.102410 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:36.102380    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20cc896b_bfb2_4e96_94cc_346b8c938a67.slice/crio-7a9844e84a0ed9aa2ed1b2193a5405fc7a36caff1a25ef0b95cc558f5b3d037d WatchSource:0}: Error finding container 7a9844e84a0ed9aa2ed1b2193a5405fc7a36caff1a25ef0b95cc558f5b3d037d: Status 404 returned error can't find the container with id 7a9844e84a0ed9aa2ed1b2193a5405fc7a36caff1a25ef0b95cc558f5b3d037d
Sep 07 16:20:36.393030 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:36.392265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7596864bf7-qnb8g" event={"ID":"20cc896b-bfb2-4e96-94cc-346b8c938a67","Type":"ContainerStarted","Data":"7a9844e84a0ed9aa2ed1b2193a5405fc7a36caff1a25ef0b95cc558f5b3d037d"}
Sep 07 16:20:36.629573 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:36.629532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-thsnm\" (UID: \"9fcc16fa-d3b8-4fee-a71c-7604f892b610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:36.632698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:36.632664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9fcc16fa-d3b8-4fee-a71c-7604f892b610-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-thsnm\" (UID: \"9fcc16fa-d3b8-4fee-a71c-7604f892b610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:36.732109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:36.732069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:38.091067 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:38.091036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"]
Sep 07 16:20:38.260282 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:38.255489    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fcc16fa_d3b8_4fee_a71c_7604f892b610.slice/crio-0b0da332bfa5b936831f656f62e13bdc4b8cc91c73fbbc58a97ca9155a71f12d WatchSource:0}: Error finding container 0b0da332bfa5b936831f656f62e13bdc4b8cc91c73fbbc58a97ca9155a71f12d: Status 404 returned error can't find the container with id 0b0da332bfa5b936831f656f62e13bdc4b8cc91c73fbbc58a97ca9155a71f12d
Sep 07 16:20:38.403883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:38.403810    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm" event={"ID":"9fcc16fa-d3b8-4fee-a71c-7604f892b610","Type":"ContainerStarted","Data":"0b0da332bfa5b936831f656f62e13bdc4b8cc91c73fbbc58a97ca9155a71f12d"}
Sep 07 16:20:38.405580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:38.405549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2" event={"ID":"32e94ad7-a852-4ee2-a4d5-81df3000856e","Type":"ContainerStarted","Data":"d85f7b29a1b053197fbe4e4e2e260174caa35f5b0a7cce7661547ae5b4d5cb03"}
Sep 07 16:20:39.425843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:39.425729    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56d5f67678-9mrz2" podStartSLOduration=2.097628299 podStartE2EDuration="5.425696046s" totalImagesPullingTime="3.328067747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:39.424894755 +0000 UTC m=+78.069190066" watchObservedRunningTime="2026-09-07 16:20:39.425696046 +0000 UTC m=+78.069991356"
Sep 07 16:20:41.421472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.421366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm" event={"ID":"9fcc16fa-d3b8-4fee-a71c-7604f892b610","Type":"ContainerStarted","Data":"fabd388afe6db762c693b11dfbad7f02df769be93ec383460493c0a635b4ef8e"}
Sep 07 16:20:41.421947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.421478    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:41.423824 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.423795    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7596864bf7-qnb8g" event={"ID":"20cc896b-bfb2-4e96-94cc-346b8c938a67","Type":"ContainerStarted","Data":"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"}
Sep 07 16:20:41.427551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.427525    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm"
Sep 07 16:20:41.463203 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.463134    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-thsnm" podStartSLOduration=4.777916943 podStartE2EDuration="7.463111677s" totalImagesPullingTime="2.685194734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:41.43727232 +0000 UTC m=+80.081567661" watchObservedRunningTime="2026-09-07 16:20:41.463111677 +0000 UTC m=+80.107406985"
Sep 07 16:20:41.463698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:41.463666    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7596864bf7-qnb8g" podStartSLOduration=1.604828669 podStartE2EDuration="6.463657799s" totalImagesPullingTime="4.85882913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:41.461996823 +0000 UTC m=+80.106292160" watchObservedRunningTime="2026-09-07 16:20:41.463657799 +0000 UTC m=+80.107953110"
Sep 07 16:20:45.394740 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:45.394619    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r9h7m"
Sep 07 16:20:45.784573 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:45.784518    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:45.784864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:45.784600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:45.791286 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:45.791256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:46.451293 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:46.451260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:20:47.356103 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.356065    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c6df6"]
Sep 07 16:20:47.360808 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.360785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.363205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.362961    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2glkw\""
Sep 07 16:20:47.363205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.362991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 16:20:47.363205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.363060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 16:20:47.363873 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.363853    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 16:20:47.363976 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.363947    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 16:20:47.429304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-root\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429484 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429319    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429484 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429402    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w65f6\" (UniqueName: \"kubernetes.io/projected/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-kube-api-access-w65f6\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429484 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-sys\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429634 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-textfile\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429634 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-tls\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429634 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429798 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-wtmp\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.429798 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.429715    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-metrics-client-ca\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531093 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-sys\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-textfile\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-sys\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-tls\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-wtmp\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-metrics-client-ca\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-root\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531503    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-wtmp\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-textfile\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.531539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.532274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-root\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.532274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.531575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w65f6\" (UniqueName: \"kubernetes.io/projected/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-kube-api-access-w65f6\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.532274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.532073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-metrics-client-ca\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.532274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.532146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.533986 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.533959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-tls\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.534098 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.534078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.538878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.538854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w65f6\" (UniqueName: \"kubernetes.io/projected/4f9b9d28-f558-4cc7-b00d-2acc1fe86042-kube-api-access-w65f6\") pod \"node-exporter-c6df6\" (UID: \"4f9b9d28-f558-4cc7-b00d-2acc1fe86042\") " pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:47.672456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:47.672361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c6df6"
Sep 07 16:20:50.346941 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:50.346909    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:20:53.473101 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:53.473069    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9b9d28_f558_4cc7_b00d_2acc1fe86042.slice/crio-9aec6096b3077e760e8809118cee88da1e674a0c4102e43e1b90753f08073196 WatchSource:0}: Error finding container 9aec6096b3077e760e8809118cee88da1e674a0c4102e43e1b90753f08073196: Status 404 returned error can't find the container with id 9aec6096b3077e760e8809118cee88da1e674a0c4102e43e1b90753f08073196
Sep 07 16:20:53.479367 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:53.479316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6df6" event={"ID":"4f9b9d28-f558-4cc7-b00d-2acc1fe86042","Type":"ContainerStarted","Data":"9aec6096b3077e760e8809118cee88da1e674a0c4102e43e1b90753f08073196"}
Sep 07 16:20:54.487226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:54.487183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zmd8z" event={"ID":"c5c51f07-797b-4b83-b922-d930a9832775","Type":"ContainerStarted","Data":"412296bceb1e80ce43ac91c61af3205395534498fc778f7b5062d76cf6975afc"}
Sep 07 16:20:54.488325 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:54.488256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:54.503982 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:54.503658    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-zmd8z" podStartSLOduration=1.655071059 podStartE2EDuration="20.503635746s" totalImagesPullingTime="18.848564687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:54.502001378 +0000 UTC m=+93.146296712" watchObservedRunningTime="2026-09-07 16:20:54.503635746 +0000 UTC m=+93.147931058"
Sep 07 16:20:54.506723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:54.506513    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-zmd8z"
Sep 07 16:20:55.494705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:55.494663    2579 generic.go:356] "Generic (PLEG): container finished" podID="4f9b9d28-f558-4cc7-b00d-2acc1fe86042" containerID="5423ef8cc0f751e737a689930f6bbda2e736818bf839a83c341b1119eb094f96" exitCode=0
Sep 07 16:20:55.495228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:55.494773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6df6" event={"ID":"4f9b9d28-f558-4cc7-b00d-2acc1fe86042","Type":"ContainerDied","Data":"5423ef8cc0f751e737a689930f6bbda2e736818bf839a83c341b1119eb094f96"}
Sep 07 16:20:56.502201 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:56.502160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6df6" event={"ID":"4f9b9d28-f558-4cc7-b00d-2acc1fe86042","Type":"ContainerStarted","Data":"e08cfb98ff3c391ab5c18c05d80415a2a1b0b847eeeac7bbdb04ed94eadda588"}
Sep 07 16:20:56.502787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:56.502221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6df6" event={"ID":"4f9b9d28-f558-4cc7-b00d-2acc1fe86042","Type":"ContainerStarted","Data":"88f08e67c32d8491bdd083470bed042858c652015283f7c20493b8647bd01c62"}
Sep 07 16:20:56.519922 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:56.519864    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c6df6" podStartSLOduration=8.041723319 podStartE2EDuration="9.51984374s" totalImagesPullingTime="1.372177641s" totalInitContainerRuntime="105.94278ms" isStatefulPod=true podCreationTimestamp="2026-09-07 16:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:20:56.518230293 +0000 UTC m=+95.162525635" watchObservedRunningTime="2026-09-07 16:20:56.51984374 +0000 UTC m=+95.164139049"
Sep 07 16:20:56.803801 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:56.803690    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:20:58.965938 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:58.965899    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:20:59.015138 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.015103    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:20:59.015350 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.015256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.023452 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.023420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 07 16:20:59.151082 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lhlr\" (UniqueName: \"kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.151467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.151403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.252842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.252842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.252842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.253125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.253125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.253125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.253125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.252995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lhlr\" (UniqueName: \"kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.253572 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.253530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.254426 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.254396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.254547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.254523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.255097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.255069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.256182 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.256155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.256327 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.256283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.262418 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.262393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lhlr\" (UniqueName: \"kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr\") pod \"console-c4db6b964-rlxj6\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") " pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.327194 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.327161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:20:59.331666 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.331634    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pc54l"
Sep 07 16:20:59.647451 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:20:59.647366    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:20:59.651729 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:20:59.651692    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77127051_51a9_44fa_80e7_721f28f4d14b.slice/crio-a286a3cca33a782ba006fba2c53caad4b92f4355ac6776151d14c0a29de29bae WatchSource:0}: Error finding container a286a3cca33a782ba006fba2c53caad4b92f4355ac6776151d14c0a29de29bae: Status 404 returned error can't find the container with id a286a3cca33a782ba006fba2c53caad4b92f4355ac6776151d14c0a29de29bae
Sep 07 16:21:00.524186 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:00.524141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4db6b964-rlxj6" event={"ID":"77127051-51a9-44fa-80e7-721f28f4d14b","Type":"ContainerStarted","Data":"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"}
Sep 07 16:21:00.524666 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:00.524252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4db6b964-rlxj6" event={"ID":"77127051-51a9-44fa-80e7-721f28f4d14b","Type":"ContainerStarted","Data":"a286a3cca33a782ba006fba2c53caad4b92f4355ac6776151d14c0a29de29bae"}
Sep 07 16:21:00.545439 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:00.545373    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c4db6b964-rlxj6" podStartSLOduration=2.545350329 podStartE2EDuration="2.545350329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:21:00.543614509 +0000 UTC m=+99.187909840" watchObservedRunningTime="2026-09-07 16:21:00.545350329 +0000 UTC m=+99.189645640"
Sep 07 16:21:09.328015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:09.327975    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:21:09.328511 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:09.328045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:21:09.332887 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:09.332859    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:21:09.566251 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:09.566222    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:21:09.607985 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:09.607901    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:21:21.830682 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:21.830626    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-89958778-plt8b" podUID="8d9c010b-ba99-4b10-8efd-195803e44aa6" containerName="registry" containerID="cri-o://bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f" gracePeriod=30
Sep 07 16:21:22.110975 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.110883    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:21:22.156057 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156029    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2ksf\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156162    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156205    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156481 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156481 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156274    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration\") pod \"8d9c010b-ba99-4b10-8efd-195803e44aa6\" (UID: \"8d9c010b-ba99-4b10-8efd-195803e44aa6\") "
Sep 07 16:21:22.156738 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156666    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:21:22.156874 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.156733    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:21:22.159344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.159290    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:21:22.159493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.159437    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf" (OuterVolumeSpecName: "kube-api-access-p2ksf") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "kube-api-access-p2ksf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:21:22.159564 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.159520    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:21:22.159564 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.159537    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:21:22.159672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.159590    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:21:22.166485 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.166455    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8d9c010b-ba99-4b10-8efd-195803e44aa6" (UID: "8d9c010b-ba99-4b10-8efd-195803e44aa6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:21:22.257449 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257411    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-bound-sa-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257449 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257444    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-image-registry-private-configuration\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257449 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257454    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2ksf\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-kube-api-access-p2ksf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257449 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257464    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257473    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d9c010b-ba99-4b10-8efd-195803e44aa6-ca-trust-extracted\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257482    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d9c010b-ba99-4b10-8efd-195803e44aa6-installation-pull-secrets\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257491    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-registry-certificates\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.257717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.257499    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d9c010b-ba99-4b10-8efd-195803e44aa6-trusted-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:22.622759 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.622721    2579 generic.go:356] "Generic (PLEG): container finished" podID="8d9c010b-ba99-4b10-8efd-195803e44aa6" containerID="bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f" exitCode=0
Sep 07 16:21:22.622927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.622799    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89958778-plt8b"
Sep 07 16:21:22.622927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.622812    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89958778-plt8b" event={"ID":"8d9c010b-ba99-4b10-8efd-195803e44aa6","Type":"ContainerDied","Data":"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"}
Sep 07 16:21:22.622927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.622877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89958778-plt8b" event={"ID":"8d9c010b-ba99-4b10-8efd-195803e44aa6","Type":"ContainerDied","Data":"ad5563de4166e1dbaf439545f39c8b4df0a524e51558ebb346c77c582004e65a"}
Sep 07 16:21:22.622927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.622905    2579 scope.go:117] "RemoveContainer" containerID="bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"
Sep 07 16:21:22.631945 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.631929    2579 scope.go:117] "RemoveContainer" containerID="bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"
Sep 07 16:21:22.632244 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:21:22.632226    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f\": container with ID starting with bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f not found: ID does not exist" containerID="bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"
Sep 07 16:21:22.632302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.632253    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f"} err="failed to get container status \"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f\": rpc error: code = NotFound desc = could not find container \"bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f\": container with ID starting with bd815b0be919dfce32093a37dc4795d6c6cf8e094246d34e81da50d1812fe06f not found: ID does not exist"
Sep 07 16:21:22.641694 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.641663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:21:22.645817 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:22.645793    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-89958778-plt8b"]
Sep 07 16:21:23.886446 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:23.886413    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9c010b-ba99-4b10-8efd-195803e44aa6" path="/var/lib/kubelet/pods/8d9c010b-ba99-4b10-8efd-195803e44aa6/volumes"
Sep 07 16:21:32.665886 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:32.665852    2579 generic.go:356] "Generic (PLEG): container finished" podID="61dd7487-9e46-4d8b-aba1-7d208e5866fc" containerID="b30123ffe937b2dccabd7261b4045613a3a39464c32b5bbafa55808e3af48d48" exitCode=0
Sep 07 16:21:32.666285 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:32.665926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c" event={"ID":"61dd7487-9e46-4d8b-aba1-7d208e5866fc","Type":"ContainerDied","Data":"b30123ffe937b2dccabd7261b4045613a3a39464c32b5bbafa55808e3af48d48"}
Sep 07 16:21:32.666285 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:32.666250    2579 scope.go:117] "RemoveContainer" containerID="b30123ffe937b2dccabd7261b4045613a3a39464c32b5bbafa55808e3af48d48"
Sep 07 16:21:33.672100 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:33.672062    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ncq5c" event={"ID":"61dd7487-9e46-4d8b-aba1-7d208e5866fc","Type":"ContainerStarted","Data":"50e97d5fba2040907fa9d7d16b1cf13109ff906aa6818d44704d972c8a11203a"}
Sep 07 16:21:34.631359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.631315    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7596864bf7-qnb8g" podUID="20cc896b-bfb2-4e96-94cc-346b8c938a67" containerName="console" containerID="cri-o://23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c" gracePeriod=15
Sep 07 16:21:34.920577 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.920540    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:21:34.959305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959271    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959308    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959331    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959377    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959402    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44brb\" (UniqueName: \"kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert\") pod \"20cc896b-bfb2-4e96-94cc-346b8c938a67\" (UID: \"20cc896b-bfb2-4e96-94cc-346b8c938a67\") "
Sep 07 16:21:34.959864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.959819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca" (OuterVolumeSpecName: "service-ca") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:21:34.960423 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.960396    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config" (OuterVolumeSpecName: "console-config") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:21:34.960423 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.960402    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:21:34.962046 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.962012    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb" (OuterVolumeSpecName: "kube-api-access-44brb") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "kube-api-access-44brb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:21:34.962377 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.962357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:21:34.962426 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:34.962387    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20cc896b-bfb2-4e96-94cc-346b8c938a67" (UID: "20cc896b-bfb2-4e96-94cc-346b8c938a67"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:21:35.060270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060239    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.060270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060265    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44brb\" (UniqueName: \"kubernetes.io/projected/20cc896b-bfb2-4e96-94cc-346b8c938a67-kube-api-access-44brb\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.060270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060276    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-oauth-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.060491 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060284    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.060491 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060296    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.060491 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.060309    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20cc896b-bfb2-4e96-94cc-346b8c938a67-console-oauth-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:21:35.681078 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.681040    2579 generic.go:356] "Generic (PLEG): container finished" podID="20cc896b-bfb2-4e96-94cc-346b8c938a67" containerID="23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c" exitCode=0
Sep 07 16:21:35.681248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.681097    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7596864bf7-qnb8g"
Sep 07 16:21:35.681248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.681108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7596864bf7-qnb8g" event={"ID":"20cc896b-bfb2-4e96-94cc-346b8c938a67","Type":"ContainerDied","Data":"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"}
Sep 07 16:21:35.681248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.681161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7596864bf7-qnb8g" event={"ID":"20cc896b-bfb2-4e96-94cc-346b8c938a67","Type":"ContainerDied","Data":"7a9844e84a0ed9aa2ed1b2193a5405fc7a36caff1a25ef0b95cc558f5b3d037d"}
Sep 07 16:21:35.681248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.681189    2579 scope.go:117] "RemoveContainer" containerID="23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"
Sep 07 16:21:35.690094 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.690067    2579 scope.go:117] "RemoveContainer" containerID="23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"
Sep 07 16:21:35.690403 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:21:35.690385    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c\": container with ID starting with 23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c not found: ID does not exist" containerID="23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"
Sep 07 16:21:35.690473 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.690411    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c"} err="failed to get container status \"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c\": rpc error: code = NotFound desc = could not find container \"23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c\": container with ID starting with 23694d80dc062dcd1abe951ee4860914394ca943da9ab51ca8c46c95fa731f0c not found: ID does not exist"
Sep 07 16:21:35.699388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.699355    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:21:35.705225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.705191    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7596864bf7-qnb8g"]
Sep 07 16:21:35.891095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:35.891059    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20cc896b-bfb2-4e96-94cc-346b8c938a67" path="/var/lib/kubelet/pods/20cc896b-bfb2-4e96-94cc-346b8c938a67/volumes"
Sep 07 16:21:37.692088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.692057    2579 generic.go:356] "Generic (PLEG): container finished" podID="fdc79fad-e548-465c-ab98-afaefef818d5" containerID="9b21f42cd87f3145846ce070bbc9d3956fc4c95dcc83f1d5605e90c9799ac9c5" exitCode=0
Sep 07 16:21:37.692543 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.692135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg" event={"ID":"fdc79fad-e548-465c-ab98-afaefef818d5","Type":"ContainerDied","Data":"9b21f42cd87f3145846ce070bbc9d3956fc4c95dcc83f1d5605e90c9799ac9c5"}
Sep 07 16:21:37.692543 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.692475    2579 scope.go:117] "RemoveContainer" containerID="9b21f42cd87f3145846ce070bbc9d3956fc4c95dcc83f1d5605e90c9799ac9c5"
Sep 07 16:21:37.693849 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.693822    2579 generic.go:356] "Generic (PLEG): container finished" podID="9fbd4161-cad8-431c-9de9-a21c2fcd6fff" containerID="a4c0c6fb005fe0fe2c614c5f64c8f4f4f26fe1f029826d83e218bde098b2f406" exitCode=0
Sep 07 16:21:37.693947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.693897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw" event={"ID":"9fbd4161-cad8-431c-9de9-a21c2fcd6fff","Type":"ContainerDied","Data":"a4c0c6fb005fe0fe2c614c5f64c8f4f4f26fe1f029826d83e218bde098b2f406"}
Sep 07 16:21:37.694191 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:37.694178    2579 scope.go:117] "RemoveContainer" containerID="a4c0c6fb005fe0fe2c614c5f64c8f4f4f26fe1f029826d83e218bde098b2f406"
Sep 07 16:21:38.700125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:38.700094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-lstgg" event={"ID":"fdc79fad-e548-465c-ab98-afaefef818d5","Type":"ContainerStarted","Data":"85475c16b153b5879ef66f90dfb7ecbb7585f344bdb090da88e7b33f4ee451d8"}
Sep 07 16:21:38.701940 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:38.701917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bsblw" event={"ID":"9fbd4161-cad8-431c-9de9-a21c2fcd6fff","Type":"ContainerStarted","Data":"b6a0f581ffb9d91c7181ad7548f7085e8cf5817ea9f56093aa74ab9462d46074"}
Sep 07 16:21:48.807260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807226    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807502    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20cc896b-bfb2-4e96-94cc-346b8c938a67" containerName="console"
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807513    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20cc896b-bfb2-4e96-94cc-346b8c938a67" containerName="console"
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807528    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9c010b-ba99-4b10-8efd-195803e44aa6" containerName="registry"
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807534    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9c010b-ba99-4b10-8efd-195803e44aa6" containerName="registry"
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807588    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20cc896b-bfb2-4e96-94cc-346b8c938a67" containerName="console"
Sep 07 16:21:48.807619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.807597    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9c010b-ba99-4b10-8efd-195803e44aa6" containerName="registry"
Sep 07 16:21:48.810730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.810701    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.820314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.820284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:21:48.871603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.871603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75p4c\" (UniqueName: \"kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.871883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.871883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.871883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.872034 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.872034 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.871931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75p4c\" (UniqueName: \"kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.973938 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.974053 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.973991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.974143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.974123    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.974198 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.974132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.975713 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.975693    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.975827 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.975764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:48.983013 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:48.982993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75p4c\" (UniqueName: \"kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c\") pod \"console-5f9887c959-778zm\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") " pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:49.123465 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:49.123354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:49.405030 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:49.404955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:21:49.408027 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:21:49.407999    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0280167b_4630_4e08_baf8_cc1779249294.slice/crio-e0a75ef91c4de7dabd72f3e8da10969a360b331da0fa084c3903cbed102f19fc WatchSource:0}: Error finding container e0a75ef91c4de7dabd72f3e8da10969a360b331da0fa084c3903cbed102f19fc: Status 404 returned error can't find the container with id e0a75ef91c4de7dabd72f3e8da10969a360b331da0fa084c3903cbed102f19fc
Sep 07 16:21:49.745137 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:49.745099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9887c959-778zm" event={"ID":"0280167b-4630-4e08-baf8-cc1779249294","Type":"ContainerStarted","Data":"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"}
Sep 07 16:21:49.745316 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:49.745157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9887c959-778zm" event={"ID":"0280167b-4630-4e08-baf8-cc1779249294","Type":"ContainerStarted","Data":"e0a75ef91c4de7dabd72f3e8da10969a360b331da0fa084c3903cbed102f19fc"}
Sep 07 16:21:49.762455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:49.762404    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f9887c959-778zm" podStartSLOduration=1.762388378 podStartE2EDuration="1.762388378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:21:49.761036788 +0000 UTC m=+148.405332099" watchObservedRunningTime="2026-09-07 16:21:49.762388378 +0000 UTC m=+148.406683731"
Sep 07 16:21:59.124123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:59.124083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:59.124539 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:59.124149    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:59.128831 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:59.128803    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:59.791037 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:59.791007    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:21:59.831411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:21:59.831376    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:22:24.858580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:24.858518    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c4db6b964-rlxj6" podUID="77127051-51a9-44fa-80e7-721f28f4d14b" containerName="console" containerID="cri-o://bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103" gracePeriod=15
Sep 07 16:22:25.094424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.094397    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:22:25.272854 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.272818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273019 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.272867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273019 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.272931    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273019 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.272968    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273019 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.272992    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273024    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lhlr\" (UniqueName: \"kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr\") pod \"77127051-51a9-44fa-80e7-721f28f4d14b\" (UID: \"77127051-51a9-44fa-80e7-721f28f4d14b\") "
Sep 07 16:22:25.273386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273258    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:22:25.273527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config" (OuterVolumeSpecName: "console-config") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:22:25.273591 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273572    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca" (OuterVolumeSpecName: "service-ca") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:22:25.273627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.273576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:22:25.275305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.275280    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr" (OuterVolumeSpecName: "kube-api-access-4lhlr") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "kube-api-access-4lhlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:22:25.275728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.275702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:22:25.275728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.275712    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "77127051-51a9-44fa-80e7-721f28f4d14b" (UID: "77127051-51a9-44fa-80e7-721f28f4d14b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:22:25.374248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374199    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-oauth-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374244    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-console-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374253    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374263    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-trusted-ca-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374495 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374271    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lhlr\" (UniqueName: \"kubernetes.io/projected/77127051-51a9-44fa-80e7-721f28f4d14b-kube-api-access-4lhlr\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374495 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374280    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77127051-51a9-44fa-80e7-721f28f4d14b-oauth-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.374495 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.374289    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77127051-51a9-44fa-80e7-721f28f4d14b-console-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:25.895465 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.895433    2579 generic.go:356] "Generic (PLEG): container finished" podID="77127051-51a9-44fa-80e7-721f28f4d14b" containerID="bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103" exitCode=0
Sep 07 16:22:25.895802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.895487    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4db6b964-rlxj6"
Sep 07 16:22:25.895802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.895490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4db6b964-rlxj6" event={"ID":"77127051-51a9-44fa-80e7-721f28f4d14b","Type":"ContainerDied","Data":"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"}
Sep 07 16:22:25.895802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.895543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4db6b964-rlxj6" event={"ID":"77127051-51a9-44fa-80e7-721f28f4d14b","Type":"ContainerDied","Data":"a286a3cca33a782ba006fba2c53caad4b92f4355ac6776151d14c0a29de29bae"}
Sep 07 16:22:25.895802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.895563    2579 scope.go:117] "RemoveContainer" containerID="bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"
Sep 07 16:22:25.903771 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.903733    2579 scope.go:117] "RemoveContainer" containerID="bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"
Sep 07 16:22:25.904046 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:22:25.904027    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103\": container with ID starting with bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103 not found: ID does not exist" containerID="bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"
Sep 07 16:22:25.904106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.904055    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103"} err="failed to get container status \"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103\": rpc error: code = NotFound desc = could not find container \"bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103\": container with ID starting with bcf283a569b32913d073c2dac157814d10ae5df799ba58104caa576c20ad3103 not found: ID does not exist"
Sep 07 16:22:25.909447 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.909422    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:22:25.914217 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:25.914194    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c4db6b964-rlxj6"]
Sep 07 16:22:27.887886 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:27.887855    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77127051-51a9-44fa-80e7-721f28f4d14b" path="/var/lib/kubelet/pods/77127051-51a9-44fa-80e7-721f28f4d14b/volumes"
Sep 07 16:22:35.970286 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.970243    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"]
Sep 07 16:22:35.970825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.970686    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77127051-51a9-44fa-80e7-721f28f4d14b" containerName="console"
Sep 07 16:22:35.970825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.970703    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="77127051-51a9-44fa-80e7-721f28f4d14b" containerName="console"
Sep 07 16:22:35.970825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.970808    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="77127051-51a9-44fa-80e7-721f28f4d14b" containerName="console"
Sep 07 16:22:35.975170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.975148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:35.978076 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.978053    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-25hs5\""
Sep 07 16:22:35.978299 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.978087    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 16:22:35.978364 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.978110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 16:22:35.979324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:35.979298    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"]
Sep 07 16:22:36.051305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.051265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.051481 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.051327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6kjh\" (UniqueName: \"kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.051481 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.051388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.152313 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.152271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.152476 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.152328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.152476 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.152379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6kjh\" (UniqueName: \"kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.152778 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.152726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.152778 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.152737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.160317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.160295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6kjh\" (UniqueName: \"kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.286455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.286366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:36.553132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.553098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"]
Sep 07 16:22:36.555834 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:22:36.555804    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec03a5a8_a573_420d_a6d7_7379add3c6a9.slice/crio-30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66 WatchSource:0}: Error finding container 30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66: Status 404 returned error can't find the container with id 30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66
Sep 07 16:22:36.937915 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:36.937823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq" event={"ID":"ec03a5a8-a573-420d-a6d7-7379add3c6a9","Type":"ContainerStarted","Data":"30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66"}
Sep 07 16:22:42.967453 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:42.967414    2579 generic.go:356] "Generic (PLEG): container finished" podID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerID="c1455fb81c576e75d2577606a82ad3f27cf9455a0d39319c522818d4d546b64d" exitCode=0
Sep 07 16:22:42.967844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:42.967499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq" event={"ID":"ec03a5a8-a573-420d-a6d7-7379add3c6a9","Type":"ContainerDied","Data":"c1455fb81c576e75d2577606a82ad3f27cf9455a0d39319c522818d4d546b64d"}
Sep 07 16:22:45.983086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:45.983046    2579 generic.go:356] "Generic (PLEG): container finished" podID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerID="da0091126b373ef948d78935c1ba36c2c6928394e9a7857870642beb51acfc27" exitCode=0
Sep 07 16:22:45.983495 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:45.983134    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq" event={"ID":"ec03a5a8-a573-420d-a6d7-7379add3c6a9","Type":"ContainerDied","Data":"da0091126b373ef948d78935c1ba36c2c6928394e9a7857870642beb51acfc27"}
Sep 07 16:22:53.016670 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:53.016631    2579 generic.go:356] "Generic (PLEG): container finished" podID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerID="d14de649df90bae366426a02ea2ba881637025fe0f24cf6177c1034d41a7e412" exitCode=0
Sep 07 16:22:53.016670 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:53.016673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq" event={"ID":"ec03a5a8-a573-420d-a6d7-7379add3c6a9","Type":"ContainerDied","Data":"d14de649df90bae366426a02ea2ba881637025fe0f24cf6177c1034d41a7e412"}
Sep 07 16:22:54.145850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.145824    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:54.209889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.209849    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util\") pod \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") "
Sep 07 16:22:54.209889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.209897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6kjh\" (UniqueName: \"kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh\") pod \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") "
Sep 07 16:22:54.210141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.209959    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle\") pod \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\" (UID: \"ec03a5a8-a573-420d-a6d7-7379add3c6a9\") "
Sep 07 16:22:54.210653 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.210629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle" (OuterVolumeSpecName: "bundle") pod "ec03a5a8-a573-420d-a6d7-7379add3c6a9" (UID: "ec03a5a8-a573-420d-a6d7-7379add3c6a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:22:54.212312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.212284    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh" (OuterVolumeSpecName: "kube-api-access-q6kjh") pod "ec03a5a8-a573-420d-a6d7-7379add3c6a9" (UID: "ec03a5a8-a573-420d-a6d7-7379add3c6a9"). InnerVolumeSpecName "kube-api-access-q6kjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:22:54.213967 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.213943    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util" (OuterVolumeSpecName: "util") pod "ec03a5a8-a573-420d-a6d7-7379add3c6a9" (UID: "ec03a5a8-a573-420d-a6d7-7379add3c6a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:22:54.310671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.310574    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:54.310671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.310606    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec03a5a8-a573-420d-a6d7-7379add3c6a9-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:54.310671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:54.310615    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6kjh\" (UniqueName: \"kubernetes.io/projected/ec03a5a8-a573-420d-a6d7-7379add3c6a9-kube-api-access-q6kjh\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:22:55.027217 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:55.027183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq" event={"ID":"ec03a5a8-a573-420d-a6d7-7379add3c6a9","Type":"ContainerDied","Data":"30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66"}
Sep 07 16:22:55.027217 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:55.027203    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406xvwq"
Sep 07 16:22:55.027447 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:55.027236    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30e0f02b19c10f4357a848e55899f6f22c30e6fccd9c1d12cb3a3c17bb648d66"
Sep 07 16:22:58.904681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904646    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"]
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904948    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="util"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904960    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="util"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904978    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="pull"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904984    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="pull"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904992    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="extract"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.904997    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="extract"
Sep 07 16:22:58.905086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.905046    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec03a5a8-a573-420d-a6d7-7379add3c6a9" containerName="extract"
Sep 07 16:22:58.921528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.921502    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:58.924570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.924546    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 07 16:22:58.925245 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.925224    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cxkkq\""
Sep 07 16:22:58.925492 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.925471    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 07 16:22:58.932099 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:58.932075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"]
Sep 07 16:22:59.050623 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.050584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89b8630b-4041-4bcd-9027-9d900ab39789-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.050879 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.050634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs464\" (UniqueName: \"kubernetes.io/projected/89b8630b-4041-4bcd-9027-9d900ab39789-kube-api-access-xs464\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.152084 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.152047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89b8630b-4041-4bcd-9027-9d900ab39789-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.152254 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.152090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs464\" (UniqueName: \"kubernetes.io/projected/89b8630b-4041-4bcd-9027-9d900ab39789-kube-api-access-xs464\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.152443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.152423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89b8630b-4041-4bcd-9027-9d900ab39789-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.162526 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.162462    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs464\" (UniqueName: \"kubernetes.io/projected/89b8630b-4041-4bcd-9027-9d900ab39789-kube-api-access-xs464\") pod \"cert-manager-operator-controller-manager-55c444bf86-lxwv5\" (UID: \"89b8630b-4041-4bcd-9027-9d900ab39789\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.232349 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.232313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"
Sep 07 16:22:59.552595 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:22:59.552559    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5"]
Sep 07 16:22:59.556730 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:22:59.556699    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b8630b_4041_4bcd_9027_9d900ab39789.slice/crio-2ccc406f965f9a1990d075baa5b81eeb5476c7bbf7bd82baf662aab032f941f6 WatchSource:0}: Error finding container 2ccc406f965f9a1990d075baa5b81eeb5476c7bbf7bd82baf662aab032f941f6: Status 404 returned error can't find the container with id 2ccc406f965f9a1990d075baa5b81eeb5476c7bbf7bd82baf662aab032f941f6
Sep 07 16:23:00.049205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:00.049170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5" event={"ID":"89b8630b-4041-4bcd-9027-9d900ab39789","Type":"ContainerStarted","Data":"2ccc406f965f9a1990d075baa5b81eeb5476c7bbf7bd82baf662aab032f941f6"}
Sep 07 16:23:03.066280 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.066226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5" event={"ID":"89b8630b-4041-4bcd-9027-9d900ab39789","Type":"ContainerStarted","Data":"a0906967367854fc60299d13b509226abf2697040d0973eb18ec80bda59c92da"}
Sep 07 16:23:03.084218 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.084149    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lxwv5" podStartSLOduration=2.531886785 podStartE2EDuration="5.084129734s" totalImagesPullingTime="2.552242949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:23:03.082833093 +0000 UTC m=+221.727128417" watchObservedRunningTime="2026-09-07 16:23:03.084129734 +0000 UTC m=+221.728425045"
Sep 07 16:23:03.680574 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.680538    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"]
Sep 07 16:23:03.696609 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.696567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"]
Sep 07 16:23:03.696793 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.696679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.699811 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.699778    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-25hs5\""
Sep 07 16:23:03.699933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.699781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 16:23:03.699933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.699781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 16:23:03.795132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.795095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2wlc\" (UniqueName: \"kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.795132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.795139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.795370 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.795161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.896024 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.895995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2wlc\" (UniqueName: \"kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.896024 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.896027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.896324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.896049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.896406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.896388    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.896482 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.896456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:03.906546 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:03.906516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2wlc\" (UniqueName: \"kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:04.008838 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:04.008794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:04.298512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:04.298455    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"]
Sep 07 16:23:04.301035 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:23:04.301007    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4516ec8d_1ce7_48ea_9598_d16dd26219b0.slice/crio-d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1 WatchSource:0}: Error finding container d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1: Status 404 returned error can't find the container with id d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1
Sep 07 16:23:05.078106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.078076    2579 generic.go:356] "Generic (PLEG): container finished" podID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerID="b76e22221f5c2444ccf938641c6f4e12b2ad0de4eb3945ba18527e65df97ef40" exitCode=0
Sep 07 16:23:05.078283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.078176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl" event={"ID":"4516ec8d-1ce7-48ea-9598-d16dd26219b0","Type":"ContainerDied","Data":"b76e22221f5c2444ccf938641c6f4e12b2ad0de4eb3945ba18527e65df97ef40"}
Sep 07 16:23:05.078283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.078231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl" event={"ID":"4516ec8d-1ce7-48ea-9598-d16dd26219b0","Type":"ContainerStarted","Data":"d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1"}
Sep 07 16:23:05.737651 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.737616    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"]
Sep 07 16:23:05.753237 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.753212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"]
Sep 07 16:23:05.753382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.753298    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:05.755497 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.755464    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fjk5m\""
Sep 07 16:23:05.755632 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.755478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 07 16:23:05.755632 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.755478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 07 16:23:05.912058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.912022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:05.912348 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:05.912089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmh4\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-kube-api-access-tnmh4\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.012637 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.012518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmh4\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-kube-api-access-tnmh4\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.012637 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.012634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.023733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.023702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.024577 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.024549    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmh4\" (UniqueName: \"kubernetes.io/projected/a9e7dadd-cf3d-4cfe-a601-f786da67d35b-kube-api-access-tnmh4\") pod \"cert-manager-cainjector-559cdd8c87-vd87f\" (UID: \"a9e7dadd-cf3d-4cfe-a601-f786da67d35b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.063117 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.063085    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"
Sep 07 16:23:06.368247 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:06.368221    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vd87f"]
Sep 07 16:23:06.370539 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:23:06.370513    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9e7dadd_cf3d_4cfe_a601_f786da67d35b.slice/crio-d869cbab64ab2f0c8c5b5a43d7ec2a6e14b8baee5de11924ceffa5d02677ef72 WatchSource:0}: Error finding container d869cbab64ab2f0c8c5b5a43d7ec2a6e14b8baee5de11924ceffa5d02677ef72: Status 404 returned error can't find the container with id d869cbab64ab2f0c8c5b5a43d7ec2a6e14b8baee5de11924ceffa5d02677ef72
Sep 07 16:23:07.088690 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:07.088657    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f" event={"ID":"a9e7dadd-cf3d-4cfe-a601-f786da67d35b","Type":"ContainerStarted","Data":"d869cbab64ab2f0c8c5b5a43d7ec2a6e14b8baee5de11924ceffa5d02677ef72"}
Sep 07 16:23:08.096388 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:08.096321    2579 generic.go:356] "Generic (PLEG): container finished" podID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerID="85f4c2f6dd479adf3b6e72e34906a542cff006019cdce1a7aa35ee82c0a683b5" exitCode=0
Sep 07 16:23:08.096878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:08.096416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl" event={"ID":"4516ec8d-1ce7-48ea-9598-d16dd26219b0","Type":"ContainerDied","Data":"85f4c2f6dd479adf3b6e72e34906a542cff006019cdce1a7aa35ee82c0a683b5"}
Sep 07 16:23:09.103199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:09.103164    2579 generic.go:356] "Generic (PLEG): container finished" podID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerID="411f9759ec6257d74a655203966befe5a2662b35aaff36ae0ff73f9937308b2c" exitCode=0
Sep 07 16:23:09.103639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:09.103248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl" event={"ID":"4516ec8d-1ce7-48ea-9598-d16dd26219b0","Type":"ContainerDied","Data":"411f9759ec6257d74a655203966befe5a2662b35aaff36ae0ff73f9937308b2c"}
Sep 07 16:23:10.109822 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.109784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f" event={"ID":"a9e7dadd-cf3d-4cfe-a601-f786da67d35b","Type":"ContainerStarted","Data":"b0b7c84710f6494f81a1a45f63d29c7b2939091e8aeccff80056a83da8bc6c36"}
Sep 07 16:23:10.126346 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.126292    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vd87f" podStartSLOduration=2.374663503 podStartE2EDuration="5.126274577s" totalImagesPullingTime="2.751611074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:23:10.125787167 +0000 UTC m=+228.770082504" watchObservedRunningTime="2026-09-07 16:23:10.126274577 +0000 UTC m=+228.770569887"
Sep 07 16:23:10.250992 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.250966    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:10.353395 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.353361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util\") pod \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") "
Sep 07 16:23:10.353555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.353431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle\") pod \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") "
Sep 07 16:23:10.353555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.353455    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2wlc\" (UniqueName: \"kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc\") pod \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\" (UID: \"4516ec8d-1ce7-48ea-9598-d16dd26219b0\") "
Sep 07 16:23:10.353852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.353829    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle" (OuterVolumeSpecName: "bundle") pod "4516ec8d-1ce7-48ea-9598-d16dd26219b0" (UID: "4516ec8d-1ce7-48ea-9598-d16dd26219b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:10.355889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.355858    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc" (OuterVolumeSpecName: "kube-api-access-m2wlc") pod "4516ec8d-1ce7-48ea-9598-d16dd26219b0" (UID: "4516ec8d-1ce7-48ea-9598-d16dd26219b0"). InnerVolumeSpecName "kube-api-access-m2wlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:23:10.358313 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.358293    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util" (OuterVolumeSpecName: "util") pod "4516ec8d-1ce7-48ea-9598-d16dd26219b0" (UID: "4516ec8d-1ce7-48ea-9598-d16dd26219b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:10.454501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.454445    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:10.454501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.454493    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4516ec8d-1ce7-48ea-9598-d16dd26219b0-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:10.454501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:10.454503    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2wlc\" (UniqueName: \"kubernetes.io/projected/4516ec8d-1ce7-48ea-9598-d16dd26219b0-kube-api-access-m2wlc\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:11.116012 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:11.115973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl" event={"ID":"4516ec8d-1ce7-48ea-9598-d16dd26219b0","Type":"ContainerDied","Data":"d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1"}
Sep 07 16:23:11.116379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:11.116023    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnp4vl"
Sep 07 16:23:11.116379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:11.116026    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7f956c7953d342ed342de97f1ac2ac6ed11686ac47b878cebc8f55273ec19c1"
Sep 07 16:23:26.551431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551393    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"]
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551710    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="util"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551724    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="util"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551735    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="pull"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551740    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="pull"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551782    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="extract"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551788    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="extract"
Sep 07 16:23:26.551875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.551866    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4516ec8d-1ce7-48ea-9598-d16dd26219b0" containerName="extract"
Sep 07 16:23:26.556262 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.556244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.559020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.558990    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 16:23:26.559020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.559009    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-25hs5\""
Sep 07 16:23:26.559354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.559327    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 16:23:26.560508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.560478    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"]
Sep 07 16:23:26.588287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.588255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.588483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.588300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.588483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.588339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bs2h\" (UniqueName: \"kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.689735 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.689686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.689735 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.689739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.690030 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.689814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bs2h\" (UniqueName: \"kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.690195 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.690172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.690257 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.690237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.697443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.697420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bs2h\" (UniqueName: \"kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:26.865829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:26.865692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:27.138158 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:27.138131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"]
Sep 07 16:23:27.140615 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:23:27.140586    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a96c5d_0bfd_4b9f_b694_12f68e23a2b1.slice/crio-607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63 WatchSource:0}: Error finding container 607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63: Status 404 returned error can't find the container with id 607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63
Sep 07 16:23:27.188768 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:27.188713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9" event={"ID":"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1","Type":"ContainerStarted","Data":"607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63"}
Sep 07 16:23:28.194722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:28.194634    2579 generic.go:356] "Generic (PLEG): container finished" podID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerID="834c3ec1b8cbe2e6cc6e2d59af2089d2d1a3601f3aab866730c7bd0d992afe53" exitCode=0
Sep 07 16:23:28.194722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:28.194705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9" event={"ID":"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1","Type":"ContainerDied","Data":"834c3ec1b8cbe2e6cc6e2d59af2089d2d1a3601f3aab866730c7bd0d992afe53"}
Sep 07 16:23:29.200521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:29.200491    2579 generic.go:356] "Generic (PLEG): container finished" podID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerID="5db0ee036e2c30b0e57c4a892ee4ea54a39eedb8ecb1aa113478ce78e0018345" exitCode=0
Sep 07 16:23:29.200921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:29.200550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9" event={"ID":"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1","Type":"ContainerDied","Data":"5db0ee036e2c30b0e57c4a892ee4ea54a39eedb8ecb1aa113478ce78e0018345"}
Sep 07 16:23:30.206479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:30.206447    2579 generic.go:356] "Generic (PLEG): container finished" podID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerID="b95fb7fdc2a74bd7da9b62fce55c7732ab29c65efef821b24b1087ce81050ff9" exitCode=0
Sep 07 16:23:30.206858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:30.206524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9" event={"ID":"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1","Type":"ContainerDied","Data":"b95fb7fdc2a74bd7da9b62fce55c7732ab29c65efef821b24b1087ce81050ff9"}
Sep 07 16:23:31.335857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.335831    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:31.430240 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.430211    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle\") pod \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") "
Sep 07 16:23:31.430408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.430272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bs2h\" (UniqueName: \"kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h\") pod \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") "
Sep 07 16:23:31.430408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.430324    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util\") pod \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\" (UID: \"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1\") "
Sep 07 16:23:31.430955 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.430926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle" (OuterVolumeSpecName: "bundle") pod "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" (UID: "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:31.432508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.432475    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h" (OuterVolumeSpecName: "kube-api-access-6bs2h") pod "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" (UID: "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1"). InnerVolumeSpecName "kube-api-access-6bs2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:23:31.436242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.436220    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util" (OuterVolumeSpecName: "util") pod "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" (UID: "14a96c5d-0bfd-4b9f-b694-12f68e23a2b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:31.531479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.531398    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:31.531479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.531427    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bs2h\" (UniqueName: \"kubernetes.io/projected/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-kube-api-access-6bs2h\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:31.531479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:31.531438    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14a96c5d-0bfd-4b9f-b694-12f68e23a2b1-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:32.217567 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:32.217536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9" event={"ID":"14a96c5d-0bfd-4b9f-b694-12f68e23a2b1","Type":"ContainerDied","Data":"607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63"}
Sep 07 16:23:32.217761 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:32.217586    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="607965a145d7b0d9af0b1d0d749423a6880ee8bd5f2964cf5f95602a1a22ad63"
Sep 07 16:23:32.217761 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:32.217550    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbfr9"
Sep 07 16:23:43.163784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.163691    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"]
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.163979    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="util"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.163991    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="util"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.164002    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="pull"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.164007    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="pull"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.164028    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="extract"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.164033    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="extract"
Sep 07 16:23:43.164241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.164081    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="14a96c5d-0bfd-4b9f-b694-12f68e23a2b1" containerName="extract"
Sep 07 16:23:43.167027 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.167010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.169096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.169076    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 16:23:43.169228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.169076    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 16:23:43.169725 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.169696    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-25hs5\""
Sep 07 16:23:43.175087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.175067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"]
Sep 07 16:23:43.233324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.233288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.233480 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.233339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmlgt\" (UniqueName: \"kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.233480 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.233401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.334344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.334304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.334524 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.334397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmlgt\" (UniqueName: \"kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.334524 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.334459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.334792 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.334770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.334839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.334792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.345868 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.345841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmlgt\" (UniqueName: \"kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.477340 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.477304    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:43.751269 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:43.751228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"]
Sep 07 16:23:43.753555 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:23:43.753519    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb072ce30_3412_459b_9632_0d242d6c9e8f.slice/crio-b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20 WatchSource:0}: Error finding container b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20: Status 404 returned error can't find the container with id b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20
Sep 07 16:23:44.271335 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.271289    2579 generic.go:356] "Generic (PLEG): container finished" podID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerID="0d102da8b255f4fd02684d3015a3755a7f42363c4dbdfebb672942a066a7f4d5" exitCode=0
Sep 07 16:23:44.271842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.271380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp" event={"ID":"b072ce30-3412-459b-9632-0d242d6c9e8f","Type":"ContainerDied","Data":"0d102da8b255f4fd02684d3015a3755a7f42363c4dbdfebb672942a066a7f4d5"}
Sep 07 16:23:44.271842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.271421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp" event={"ID":"b072ce30-3412-459b-9632-0d242d6c9e8f","Type":"ContainerStarted","Data":"b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20"}
Sep 07 16:23:44.499605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.499572    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"]
Sep 07 16:23:44.502387 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.502361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.504509 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.504488    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f6lx2\""
Sep 07 16:23:44.504980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.504960    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 16:23:44.505101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.504960    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 16:23:44.505101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.505007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 16:23:44.505101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.504960    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 16:23:44.518467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.518435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"]
Sep 07 16:23:44.545762 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.545654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.545762 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.545692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r95fl\" (UniqueName: \"kubernetes.io/projected/dbee4a3c-3a98-4dfb-a599-b96c4783927d-kube-api-access-r95fl\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.545951 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.545790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-tmp\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.545951 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.545831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.545951 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.545852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-manifests\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.646908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.646875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.647089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.646915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r95fl\" (UniqueName: \"kubernetes.io/projected/dbee4a3c-3a98-4dfb-a599-b96c4783927d-kube-api-access-r95fl\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.647089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.646954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-tmp\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.647089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.646979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.647089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.646995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-manifests\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.647450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.647430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-manifests\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.649607 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.649579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbee4a3c-3a98-4dfb-a599-b96c4783927d-tmp\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.649840 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.649802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.649840 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.649816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbee4a3c-3a98-4dfb-a599-b96c4783927d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.657920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.657895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r95fl\" (UniqueName: \"kubernetes.io/projected/dbee4a3c-3a98-4dfb-a599-b96c4783927d-kube-api-access-r95fl\") pod \"opendatahub-operator-controller-manager-7766cc5489-77jsf\" (UID: \"dbee4a3c-3a98-4dfb-a599-b96c4783927d\") " pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:44.812784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:44.812759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:45.092458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:45.092432    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"]
Sep 07 16:23:45.094942 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:23:45.094914    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbee4a3c_3a98_4dfb_a599_b96c4783927d.slice/crio-9afa660596f14490fdb08709a79f82335d02cabdea9ab2e29d9b7f79c18f6dcb WatchSource:0}: Error finding container 9afa660596f14490fdb08709a79f82335d02cabdea9ab2e29d9b7f79c18f6dcb: Status 404 returned error can't find the container with id 9afa660596f14490fdb08709a79f82335d02cabdea9ab2e29d9b7f79c18f6dcb
Sep 07 16:23:45.276874 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:45.276830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf" event={"ID":"dbee4a3c-3a98-4dfb-a599-b96c4783927d","Type":"ContainerStarted","Data":"9afa660596f14490fdb08709a79f82335d02cabdea9ab2e29d9b7f79c18f6dcb"}
Sep 07 16:23:45.278703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:45.278675    2579 generic.go:356] "Generic (PLEG): container finished" podID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerID="b4050827da429bd08db7d2ddc26dd398d4dd4560b79f1692ca897b0e18aacc5a" exitCode=0
Sep 07 16:23:45.278835 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:45.278785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp" event={"ID":"b072ce30-3412-459b-9632-0d242d6c9e8f","Type":"ContainerDied","Data":"b4050827da429bd08db7d2ddc26dd398d4dd4560b79f1692ca897b0e18aacc5a"}
Sep 07 16:23:46.286197 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:46.286161    2579 generic.go:356] "Generic (PLEG): container finished" podID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerID="d982e30cb59ceedfbfd3db2b9d283d440a9297b820efabca34b07de8ba8f14c8" exitCode=0
Sep 07 16:23:46.286556 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:46.286216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp" event={"ID":"b072ce30-3412-459b-9632-0d242d6c9e8f","Type":"ContainerDied","Data":"d982e30cb59ceedfbfd3db2b9d283d440a9297b820efabca34b07de8ba8f14c8"}
Sep 07 16:23:47.433997 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.433969    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:47.572405 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.572247    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util\") pod \"b072ce30-3412-459b-9632-0d242d6c9e8f\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") "
Sep 07 16:23:47.572405 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.572323    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle\") pod \"b072ce30-3412-459b-9632-0d242d6c9e8f\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") "
Sep 07 16:23:47.572405 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.572374    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmlgt\" (UniqueName: \"kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt\") pod \"b072ce30-3412-459b-9632-0d242d6c9e8f\" (UID: \"b072ce30-3412-459b-9632-0d242d6c9e8f\") "
Sep 07 16:23:47.573151 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.573128    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle" (OuterVolumeSpecName: "bundle") pod "b072ce30-3412-459b-9632-0d242d6c9e8f" (UID: "b072ce30-3412-459b-9632-0d242d6c9e8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:47.575288 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.575253    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt" (OuterVolumeSpecName: "kube-api-access-zmlgt") pod "b072ce30-3412-459b-9632-0d242d6c9e8f" (UID: "b072ce30-3412-459b-9632-0d242d6c9e8f"). InnerVolumeSpecName "kube-api-access-zmlgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:23:47.578355 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.578330    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util" (OuterVolumeSpecName: "util") pod "b072ce30-3412-459b-9632-0d242d6c9e8f" (UID: "b072ce30-3412-459b-9632-0d242d6c9e8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:23:47.673174 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.673129    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmlgt\" (UniqueName: \"kubernetes.io/projected/b072ce30-3412-459b-9632-0d242d6c9e8f-kube-api-access-zmlgt\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:47.673174 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.673176    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:47.673420 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:47.673191    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b072ce30-3412-459b-9632-0d242d6c9e8f-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:23:48.304304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:48.304276    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp"
Sep 07 16:23:48.304544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:48.304278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0g2nxp" event={"ID":"b072ce30-3412-459b-9632-0d242d6c9e8f","Type":"ContainerDied","Data":"b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20"}
Sep 07 16:23:48.304544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:48.304433    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b76149196fa25038595ce9568d9089e424bbe514962d9650b2dd756966916e20"
Sep 07 16:23:49.310826 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:49.310789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf" event={"ID":"dbee4a3c-3a98-4dfb-a599-b96c4783927d","Type":"ContainerStarted","Data":"e6d3404a772016ab5e88b41c3c6de587ea994231ee134a11aa889da9ca51bd96"}
Sep 07 16:23:50.316584 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:50.316547    2579 generic.go:356] "Generic (PLEG): container finished" podID="dbee4a3c-3a98-4dfb-a599-b96c4783927d" containerID="e6d3404a772016ab5e88b41c3c6de587ea994231ee134a11aa889da9ca51bd96" exitCode=0
Sep 07 16:23:50.317021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:50.316641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf" event={"ID":"dbee4a3c-3a98-4dfb-a599-b96c4783927d","Type":"ContainerDied","Data":"e6d3404a772016ab5e88b41c3c6de587ea994231ee134a11aa889da9ca51bd96"}
Sep 07 16:23:51.323411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:51.323322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf" event={"ID":"dbee4a3c-3a98-4dfb-a599-b96c4783927d","Type":"ContainerStarted","Data":"7ef556dba15949cd31fd9a4b35182b3b54afe08b949d0b936351bf5e1f645f4f"}
Sep 07 16:23:51.323781 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:51.323456    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:23:51.342330 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:23:51.342272    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf" podStartSLOduration=2.307162339 podStartE2EDuration="7.342256201s" totalImagesPullingTime="3.994416249s" totalInitContainerRuntime="1.040677613s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:23:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:23:51.340137126 +0000 UTC m=+269.984432446" watchObservedRunningTime="2026-09-07 16:23:51.342256201 +0000 UTC m=+269.986551511"
Sep 07 16:24:02.330908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:02.330876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7766cc5489-77jsf"
Sep 07 16:24:04.232308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232269    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"]
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232572    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="pull"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232584    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="pull"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232605    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="util"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232610    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="util"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232616    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="extract"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232621    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="extract"
Sep 07 16:24:04.232681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.232672    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b072ce30-3412-459b-9632-0d242d6c9e8f" containerName="extract"
Sep 07 16:24:04.240673 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.240648    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.243092 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.243062    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8xchv\""
Sep 07 16:24:04.243212 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.243101    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 16:24:04.243593 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.243575    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 16:24:04.248808 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.248787    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"]
Sep 07 16:24:04.418493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.418454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384721ba-d51f-4bc9-9d85-16b83d60fec1-tmp\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.418681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.418519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k54rn\" (UniqueName: \"kubernetes.io/projected/384721ba-d51f-4bc9-9d85-16b83d60fec1-kube-api-access-k54rn\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.418681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.418557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.519310 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.519217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384721ba-d51f-4bc9-9d85-16b83d60fec1-tmp\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.519310 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.519280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k54rn\" (UniqueName: \"kubernetes.io/projected/384721ba-d51f-4bc9-9d85-16b83d60fec1-kube-api-access-k54rn\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.519541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.519312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.519541 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:04.519403    2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 07 16:24:04.519541 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:04.519457    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs podName:384721ba-d51f-4bc9-9d85-16b83d60fec1 nodeName:}" failed. No retries permitted until 2026-09-07 16:24:05.019441963 +0000 UTC m=+283.663737251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs") pod "kube-auth-proxy-6797cb654d-2cd9b" (UID: "384721ba-d51f-4bc9-9d85-16b83d60fec1") : secret "kube-auth-proxy-tls" not found
Sep 07 16:24:04.521619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.521594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384721ba-d51f-4bc9-9d85-16b83d60fec1-tmp\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:04.527216 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:04.527191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k54rn\" (UniqueName: \"kubernetes.io/projected/384721ba-d51f-4bc9-9d85-16b83d60fec1-kube-api-access-k54rn\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:05.024601 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.024559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:05.027230 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.027201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/384721ba-d51f-4bc9-9d85-16b83d60fec1-tls-certs\") pod \"kube-auth-proxy-6797cb654d-2cd9b\" (UID: \"384721ba-d51f-4bc9-9d85-16b83d60fec1\") " pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:05.151659 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.151620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"
Sep 07 16:24:05.426961 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.426937    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:05.428716 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:05.428679    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384721ba_d51f_4bc9_9d85_16b83d60fec1.slice/crio-ca55bea75940c888e74e3eabdb81aa0cce1132fcec00e0bc85f7e4958e3af96b WatchSource:0}: Error finding container ca55bea75940c888e74e3eabdb81aa0cce1132fcec00e0bc85f7e4958e3af96b: Status 404 returned error can't find the container with id ca55bea75940c888e74e3eabdb81aa0cce1132fcec00e0bc85f7e4958e3af96b
Sep 07 16:24:05.431443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.431425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b"]
Sep 07 16:24:05.431537 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.431510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.433619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.433598    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 07 16:24:05.433730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.433638    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 16:24:05.433730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.433669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fc7kl\""
Sep 07 16:24:05.433730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.433678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 07 16:24:05.433730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.433713    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 07 16:24:05.442431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.442411    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:05.527624 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.527816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.527816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.527816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.527816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97m4h\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.528025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.528025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.527919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.628883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.628851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.628893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m4h\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.628912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.628931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629212 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.629066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629212 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.629135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629212 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.629168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.629608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.629581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.631581 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.631548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.631671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.631578    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.631950 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.631931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.631997 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.631966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.640838 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.640812    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.640921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.640837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97m4h\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h\") pod \"istiod-openshift-gateway-7b7995d5cd-r7blb\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:05.741818 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:05.741787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:06.015555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:06.015441    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:06.017518 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:06.017466    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9355f6fb_276a_47e2_b814_0a645e196901.slice/crio-b6ba055cf7cce09b37e082e7a9720cc6edab7094f5bcbde70902d2731040119e WatchSource:0}: Error finding container b6ba055cf7cce09b37e082e7a9720cc6edab7094f5bcbde70902d2731040119e: Status 404 returned error can't find the container with id b6ba055cf7cce09b37e082e7a9720cc6edab7094f5bcbde70902d2731040119e
Sep 07 16:24:06.400819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:06.400714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b" event={"ID":"384721ba-d51f-4bc9-9d85-16b83d60fec1","Type":"ContainerStarted","Data":"ca55bea75940c888e74e3eabdb81aa0cce1132fcec00e0bc85f7e4958e3af96b"}
Sep 07 16:24:06.402717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:06.402684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" event={"ID":"9355f6fb-276a-47e2-b814-0a645e196901","Type":"ContainerStarted","Data":"b6ba055cf7cce09b37e082e7a9720cc6edab7094f5bcbde70902d2731040119e"}
Sep 07 16:24:08.416467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:08.416128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b" event={"ID":"384721ba-d51f-4bc9-9d85-16b83d60fec1","Type":"ContainerStarted","Data":"652a7fccb46efeb36b807fb9c0d8bb6d26a90039904dfaa5c751fe502130706c"}
Sep 07 16:24:08.431967 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:08.431911    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6797cb654d-2cd9b" podStartSLOduration=1.771863966 podStartE2EDuration="4.431890507s" totalImagesPullingTime="2.660026541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:08.430810823 +0000 UTC m=+287.075106127" watchObservedRunningTime="2026-09-07 16:24:08.431890507 +0000 UTC m=+287.076185818"
Sep 07 16:24:09.895562 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:09.895525    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:24:09.895839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:09.895609    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:24:10.431347 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:10.431274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" event={"ID":"9355f6fb-276a-47e2-b814-0a645e196901","Type":"ContainerStarted","Data":"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"}
Sep 07 16:24:10.431646 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:10.431610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:10.434072 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:10.434026    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-r7blb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 16:24:10.434207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:10.434107    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" podUID="9355f6fb-276a-47e2-b814-0a645e196901" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 16:24:10.451602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:10.451537    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" podStartSLOduration=1.576687739 podStartE2EDuration="5.451514149s" totalImagesPullingTime="3.87482641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:10.450150311 +0000 UTC m=+289.094445681" watchObservedRunningTime="2026-09-07 16:24:10.451514149 +0000 UTC m=+289.095809460"
Sep 07 16:24:11.437196 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:11.437165    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:14.812181 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.812144    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwjtb"]
Sep 07 16:24:14.815312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.815295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:14.817533 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.817502    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 07 16:24:14.817660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.817595    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kpmh2\""
Sep 07 16:24:14.822035 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.822007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwjtb"]
Sep 07 16:24:14.910397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.910362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9ggg\" (UniqueName: \"kubernetes.io/projected/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-kube-api-access-f9ggg\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:14.910397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:14.910400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.011518 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.011484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.011688 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:15.011641    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 07 16:24:15.011688 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.011667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9ggg\" (UniqueName: \"kubernetes.io/projected/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-kube-api-access-f9ggg\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.011842 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:15.011726    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert podName:7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9 nodeName:}" failed. No retries permitted until 2026-09-07 16:24:15.511703897 +0000 UTC m=+294.155999203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert") pod "odh-model-controller-7998c67dc4-cwjtb" (UID: "7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9") : secret "odh-model-controller-webhook-cert" not found
Sep 07 16:24:15.019719 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.019694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9ggg\" (UniqueName: \"kubernetes.io/projected/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-kube-api-access-f9ggg\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.416187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.416150    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n9xgx"]
Sep 07 16:24:15.419326 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.419307    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.421211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.421189    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 07 16:24:15.421337 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.421275    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-tzfzv\""
Sep 07 16:24:15.428725 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.428700    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n9xgx"]
Sep 07 16:24:15.515103 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.515067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.515267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.515130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g9n7\" (UniqueName: \"kubernetes.io/projected/e77947cc-2180-414c-b67c-f5e49a013789-kube-api-access-4g9n7\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.515267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.515161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.517784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.517732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9-cert\") pod \"odh-model-controller-7998c67dc4-cwjtb\" (UID: \"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9\") " pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.615780 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.615712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g9n7\" (UniqueName: \"kubernetes.io/projected/e77947cc-2180-414c-b67c-f5e49a013789-kube-api-access-4g9n7\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.615944 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.615788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.615944 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:15.615878    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 07 16:24:15.615944 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:15.615928    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs podName:e77947cc-2180-414c-b67c-f5e49a013789 nodeName:}" failed. No retries permitted until 2026-09-07 16:24:16.11591319 +0000 UTC m=+294.760208478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs") pod "model-serving-api-54d89bd5fc-n9xgx" (UID: "e77947cc-2180-414c-b67c-f5e49a013789") : secret "model-serving-api-tls" not found
Sep 07 16:24:15.623944 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.623919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g9n7\" (UniqueName: \"kubernetes.io/projected/e77947cc-2180-414c-b67c-f5e49a013789-kube-api-access-4g9n7\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:15.728106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.728077    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:15.997856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:15.997827    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cwjtb"]
Sep 07 16:24:16.000117 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:16.000077    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af43cc6_f6e9_4c2c_999f_ae32eadaa6d9.slice/crio-ceeea5f70c3cb819e3539b6d4aed81b64cdaa99cc10d680f3b909f6206a0fee2 WatchSource:0}: Error finding container ceeea5f70c3cb819e3539b6d4aed81b64cdaa99cc10d680f3b909f6206a0fee2: Status 404 returned error can't find the container with id ceeea5f70c3cb819e3539b6d4aed81b64cdaa99cc10d680f3b909f6206a0fee2
Sep 07 16:24:16.117815 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:16.117779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:16.120352 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:16.120327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e77947cc-2180-414c-b67c-f5e49a013789-tls-certs\") pod \"model-serving-api-54d89bd5fc-n9xgx\" (UID: \"e77947cc-2180-414c-b67c-f5e49a013789\") " pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:16.331124 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:16.331043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:16.461926 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:16.461893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb" event={"ID":"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9","Type":"ContainerStarted","Data":"ceeea5f70c3cb819e3539b6d4aed81b64cdaa99cc10d680f3b909f6206a0fee2"}
Sep 07 16:24:16.604758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:16.604717    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n9xgx"]
Sep 07 16:24:16.606793 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:16.606737    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77947cc_2180_414c_b67c_f5e49a013789.slice/crio-54c64011f56d0c6d1f49d60a05c8e45c59d140e3cef5033fa60a8a5a4614faf0 WatchSource:0}: Error finding container 54c64011f56d0c6d1f49d60a05c8e45c59d140e3cef5033fa60a8a5a4614faf0: Status 404 returned error can't find the container with id 54c64011f56d0c6d1f49d60a05c8e45c59d140e3cef5033fa60a8a5a4614faf0
Sep 07 16:24:17.470282 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:17.470235    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx" event={"ID":"e77947cc-2180-414c-b67c-f5e49a013789","Type":"ContainerStarted","Data":"54c64011f56d0c6d1f49d60a05c8e45c59d140e3cef5033fa60a8a5a4614faf0"}
Sep 07 16:24:19.484048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.483985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb" event={"ID":"7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9","Type":"ContainerStarted","Data":"4e8e910cde1dfbf805a0a573b452de8766b65f160609142c607c8845cfd34fb1"}
Sep 07 16:24:19.484521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.484096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:19.485931 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.485894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx" event={"ID":"e77947cc-2180-414c-b67c-f5e49a013789","Type":"ContainerStarted","Data":"47f1c834cf3d810c189be4063f24044fadfe51b71f2355aabc84faec66dc1a51"}
Sep 07 16:24:19.486053 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.485977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:19.504437 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.504379    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb" podStartSLOduration=2.409439062 podStartE2EDuration="5.50436607s" totalImagesPullingTime="3.094927008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:19.502640356 +0000 UTC m=+298.146935704" watchObservedRunningTime="2026-09-07 16:24:19.50436607 +0000 UTC m=+298.148661379"
Sep 07 16:24:19.519306 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:19.519256    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx" podStartSLOduration=2.034693039 podStartE2EDuration="4.519242991s" totalImagesPullingTime="2.484549952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:19.517363915 +0000 UTC m=+298.161659248" watchObservedRunningTime="2026-09-07 16:24:19.519242991 +0000 UTC m=+298.163538381"
Sep 07 16:24:21.794058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:21.794029    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:24:21.794518 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:21.794263    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:24:21.805077 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:21.805052    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 16:24:24.049636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.049588    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"]
Sep 07 16:24:24.054408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.054381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.059171 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.059152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 07 16:24:24.059786 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.059761    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-85vxg\""
Sep 07 16:24:24.062606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.062038    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"]
Sep 07 16:24:24.067624 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.067603    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"]
Sep 07 16:24:24.067728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.067691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.070651 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.070631    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 07 16:24:24.074012 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.073993    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5t5cc\""
Sep 07 16:24:24.084260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.084235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d2e8250-239b-415a-ace9-31be31c0a7b9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.084386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.084303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpbsw\" (UniqueName: \"kubernetes.io/projected/2ba65f54-d03d-45ab-8a2b-64e68250d677-kube-api-access-xpbsw\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.084386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.084342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr7jq\" (UniqueName: \"kubernetes.io/projected/9d2e8250-239b-415a-ace9-31be31c0a7b9-kube-api-access-sr7jq\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.084480 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.084427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.098866 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.098816    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"]
Sep 07 16:24:24.185534 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.185486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d2e8250-239b-415a-ace9-31be31c0a7b9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.185741 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.185566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpbsw\" (UniqueName: \"kubernetes.io/projected/2ba65f54-d03d-45ab-8a2b-64e68250d677-kube-api-access-xpbsw\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.185741 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.185594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7jq\" (UniqueName: \"kubernetes.io/projected/9d2e8250-239b-415a-ace9-31be31c0a7b9-kube-api-access-sr7jq\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.185741 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.185648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.185741 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:24.185763    2579 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 07 16:24:24.185986 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:24.185833    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert podName:2ba65f54-d03d-45ab-8a2b-64e68250d677 nodeName:}" failed. No retries permitted until 2026-09-07 16:24:24.685810899 +0000 UTC m=+303.330106204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert") pod "kserve-controller-manager-d6f4b65b8-cs4vd" (UID: "2ba65f54-d03d-45ab-8a2b-64e68250d677") : secret "kserve-webhook-server-cert" not found
Sep 07 16:24:24.188443 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.188411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d2e8250-239b-415a-ace9-31be31c0a7b9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.195893 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.195869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr7jq\" (UniqueName: \"kubernetes.io/projected/9d2e8250-239b-415a-ace9-31be31c0a7b9-kube-api-access-sr7jq\") pod \"llmisvc-controller-manager-777c6bd7d5-5qnzr\" (UID: \"9d2e8250-239b-415a-ace9-31be31c0a7b9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.196700 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.196682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpbsw\" (UniqueName: \"kubernetes.io/projected/2ba65f54-d03d-45ab-8a2b-64e68250d677-kube-api-access-xpbsw\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.378361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.378270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:24.652338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.652313    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"]
Sep 07 16:24:24.654605 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:24.654569    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9d2e8250_239b_415a_ace9_31be31c0a7b9.slice/crio-043bf16f3216494dac8c01104d9d9ee50f1841e3922adea5072435224653aaac WatchSource:0}: Error finding container 043bf16f3216494dac8c01104d9d9ee50f1841e3922adea5072435224653aaac: Status 404 returned error can't find the container with id 043bf16f3216494dac8c01104d9d9ee50f1841e3922adea5072435224653aaac
Sep 07 16:24:24.656106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.656089    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:24:24.690226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.690186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.692726 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.692707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ba65f54-d03d-45ab-8a2b-64e68250d677-cert\") pod \"kserve-controller-manager-d6f4b65b8-cs4vd\" (UID: \"2ba65f54-d03d-45ab-8a2b-64e68250d677\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:24.967889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:24.967851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:25.243835 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:25.243789    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"]
Sep 07 16:24:25.246177 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:24:25.246148    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba65f54_d03d_45ab_8a2b_64e68250d677.slice/crio-c36c51565efc1e39ab1d2810e3fc0f9dabe6f5df6a141c384185240a072d560b WatchSource:0}: Error finding container c36c51565efc1e39ab1d2810e3fc0f9dabe6f5df6a141c384185240a072d560b: Status 404 returned error can't find the container with id c36c51565efc1e39ab1d2810e3fc0f9dabe6f5df6a141c384185240a072d560b
Sep 07 16:24:25.515944 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:25.515864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd" event={"ID":"2ba65f54-d03d-45ab-8a2b-64e68250d677","Type":"ContainerStarted","Data":"c36c51565efc1e39ab1d2810e3fc0f9dabe6f5df6a141c384185240a072d560b"}
Sep 07 16:24:25.517173 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:25.517146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr" event={"ID":"9d2e8250-239b-415a-ace9-31be31c0a7b9","Type":"ContainerStarted","Data":"043bf16f3216494dac8c01104d9d9ee50f1841e3922adea5072435224653aaac"}
Sep 07 16:24:25.776814 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:25.776723    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:25.776958 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:25.776931    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" podUID="9355f6fb-276a-47e2-b814-0a645e196901" containerName="discovery" containerID="cri-o://4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec" gracePeriod=30
Sep 07 16:24:26.246653 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.246370    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:26.304902 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.304795    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.304902 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.304855    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.304919    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97m4h\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.304954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.305004    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.305053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.305088    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs\") pod \"9355f6fb-276a-47e2-b814-0a645e196901\" (UID: \"9355f6fb-276a-47e2-b814-0a645e196901\") "
Sep 07 16:24:26.305974 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.305940    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:24:26.308682 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.308616    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts" (OuterVolumeSpecName: "cacerts") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:24:26.308862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.308822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:24:26.309572 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.309535    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:24:26.309961 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.309927    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs" (OuterVolumeSpecName: "local-certs") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:24:26.310324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.310295    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h" (OuterVolumeSpecName: "kube-api-access-97m4h") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "kube-api-access-97m4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:24:26.310396 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.310376    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token" (OuterVolumeSpecName: "istio-token") pod "9355f6fb-276a-47e2-b814-0a645e196901" (UID: "9355f6fb-276a-47e2-b814-0a645e196901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:24:26.406478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406441    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97m4h\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-kube-api-access-97m4h\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406476    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-cacerts\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406489    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-ca-configmap\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406501    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-kubeconfig\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406513    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9355f6fb-276a-47e2-b814-0a645e196901-local-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406523    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9355f6fb-276a-47e2-b814-0a645e196901-istio-token\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.406858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.406535    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9355f6fb-276a-47e2-b814-0a645e196901-istio-csr-dns-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:24:26.525767 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.525713    2579 generic.go:356] "Generic (PLEG): container finished" podID="9355f6fb-276a-47e2-b814-0a645e196901" containerID="4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec" exitCode=0
Sep 07 16:24:26.526009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.525886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" event={"ID":"9355f6fb-276a-47e2-b814-0a645e196901","Type":"ContainerDied","Data":"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"}
Sep 07 16:24:26.526009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.525935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb" event={"ID":"9355f6fb-276a-47e2-b814-0a645e196901","Type":"ContainerDied","Data":"b6ba055cf7cce09b37e082e7a9720cc6edab7094f5bcbde70902d2731040119e"}
Sep 07 16:24:26.526009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.525961    2579 scope.go:117] "RemoveContainer" containerID="4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"
Sep 07 16:24:26.526191 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.526121    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"
Sep 07 16:24:26.544373 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.544342    2579 scope.go:117] "RemoveContainer" containerID="4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"
Sep 07 16:24:26.544727 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:24:26.544696    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec\": container with ID starting with 4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec not found: ID does not exist" containerID="4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"
Sep 07 16:24:26.544860 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.544734    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec"} err="failed to get container status \"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec\": rpc error: code = NotFound desc = could not find container \"4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec\": container with ID starting with 4411ab085cb17be7af73d78b655d7194eae0fa36b16d2861096339f41a87bcec not found: ID does not exist"
Sep 07 16:24:26.550489 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.550464    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:26.554836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:26.554807    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7blb"]
Sep 07 16:24:27.889949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:27.889885    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9355f6fb-276a-47e2-b814-0a645e196901" path="/var/lib/kubelet/pods/9355f6fb-276a-47e2-b814-0a645e196901/volumes"
Sep 07 16:24:28.539126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.539091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd" event={"ID":"2ba65f54-d03d-45ab-8a2b-64e68250d677","Type":"ContainerStarted","Data":"3d7c2f28937546ca387d541babe60d9d7c2996150bd6285335bb9b04acc928d7"}
Sep 07 16:24:28.539318 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.539194    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:24:28.540847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.540817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr" event={"ID":"9d2e8250-239b-415a-ace9-31be31c0a7b9","Type":"ContainerStarted","Data":"490ec1296921f7764c1559956a10df00796bd915bcadc43bc24f7076e89bc516"}
Sep 07 16:24:28.540954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.540866    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:28.557306 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.557253    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd" podStartSLOduration=1.747353814 podStartE2EDuration="4.557237885s" totalImagesPullingTime="2.809884071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:28.555407282 +0000 UTC m=+307.199702628" watchObservedRunningTime="2026-09-07 16:24:28.557237885 +0000 UTC m=+307.201533196"
Sep 07 16:24:28.572390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:28.572337    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr" podStartSLOduration=1.166573277 podStartE2EDuration="4.57232062s" totalImagesPullingTime="3.405747343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:24:28.570005933 +0000 UTC m=+307.214301261" watchObservedRunningTime="2026-09-07 16:24:28.57232062 +0000 UTC m=+307.216615930"
Sep 07 16:24:30.492656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:30.492624    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cwjtb"
Sep 07 16:24:30.494519 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:30.494494    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-n9xgx"
Sep 07 16:24:59.547622 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:59.547588    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5qnzr"
Sep 07 16:24:59.551231 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:24:59.551204    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cs4vd"
Sep 07 16:25:19.056854 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.056772    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"]
Sep 07 16:25:19.057272 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.057108    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9355f6fb-276a-47e2-b814-0a645e196901" containerName="discovery"
Sep 07 16:25:19.057272 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.057118    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9355f6fb-276a-47e2-b814-0a645e196901" containerName="discovery"
Sep 07 16:25:19.057272 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.057182    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9355f6fb-276a-47e2-b814-0a645e196901" containerName="discovery"
Sep 07 16:25:19.060317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.060290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.062382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.062363    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 16:25:19.062466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.062396    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-25hs5\""
Sep 07 16:25:19.063704 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.063532    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 16:25:19.072634 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.072607    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"]
Sep 07 16:25:19.139438 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.139401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sh5f\" (UniqueName: \"kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.139630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.139543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.139699 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.139668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.154848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.154818    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"]
Sep 07 16:25:19.158424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.158403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.164520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.164497    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"]
Sep 07 16:25:19.240098 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nxqm\" (UniqueName: \"kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.240098 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.240314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.240314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sh5f\" (UniqueName: \"kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.240314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.240314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.240501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.240545 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.240530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.252469 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.252447    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sh5f\" (UniqueName: \"kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.258259 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.258236    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"]
Sep 07 16:25:19.262404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.262389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.268105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.268082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"]
Sep 07 16:25:19.340791 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.340791 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.340791 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.341066 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhblv\" (UniqueName: \"kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.341066 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nxqm\" (UniqueName: \"kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.341066 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.340895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.341189 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.341077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.341189 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.341179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.351096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.351067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nxqm\" (UniqueName: \"kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.359579 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.359558    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"]
Sep 07 16:25:19.363105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.363091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.370718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.370663    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"]
Sep 07 16:25:19.375353 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.375332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:19.441455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441425    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.441603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crkk6\" (UniqueName: \"kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.441603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.441603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.441790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhblv\" (UniqueName: \"kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.441790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.441996 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.442095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.441963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.450208 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.450186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhblv\" (UniqueName: \"kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.468648 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.468627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:19.543131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.542411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crkk6\" (UniqueName: \"kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.543131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.542558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.543131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.542604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.543131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.543080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.543440 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.543271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.551144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.551120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crkk6\" (UniqueName: \"kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.573005 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.572980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:19.647543 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.647518    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"]
Sep 07 16:25:19.649404 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:19.649379    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8749fe_f7a8_4dab_b969_469eb428bad0.slice/crio-e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546 WatchSource:0}: Error finding container e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546: Status 404 returned error can't find the container with id e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546
Sep 07 16:25:19.673339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.673317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:19.739563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.739536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"]
Sep 07 16:25:19.741004 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:19.740979    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11014ce_b72e_479b_b461_2a59d6be9bb1.slice/crio-419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5 WatchSource:0}: Error finding container 419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5: Status 404 returned error can't find the container with id 419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5
Sep 07 16:25:19.792729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.792697    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerID="cc6ef7f82ffc80976f9634d403da2f50a4639e9981826ac7f126b1b311653911" exitCode=0
Sep 07 16:25:19.792885 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.792786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6" event={"ID":"9d8749fe-f7a8-4dab-b969-469eb428bad0","Type":"ContainerDied","Data":"cc6ef7f82ffc80976f9634d403da2f50a4639e9981826ac7f126b1b311653911"}
Sep 07 16:25:19.792885 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.792838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6" event={"ID":"9d8749fe-f7a8-4dab-b969-469eb428bad0","Type":"ContainerStarted","Data":"e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546"}
Sep 07 16:25:19.794130 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.794107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g" event={"ID":"a11014ce-b72e-479b-b461-2a59d6be9bb1","Type":"ContainerStarted","Data":"419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5"}
Sep 07 16:25:19.845244 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.845114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"]
Sep 07 16:25:19.846822 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:19.846716    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab701f6_6926_4d46_a19b_5297fc9babcc.slice/crio-7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994 WatchSource:0}: Error finding container 7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994: Status 404 returned error can't find the container with id 7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994
Sep 07 16:25:19.945975 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:19.945933    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"]
Sep 07 16:25:19.949258 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:19.949225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e78264_6c59_4e5d_9983_ca8b08eb27cc.slice/crio-786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0 WatchSource:0}: Error finding container 786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0: Status 404 returned error can't find the container with id 786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0
Sep 07 16:25:20.805555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.805529    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerID="7d030940dac13594c6ec234217429a9145b55a445ab7b81934e33a21c7e23f93" exitCode=0
Sep 07 16:25:20.805916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.805615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw" event={"ID":"6ab701f6-6926-4d46-a19b-5297fc9babcc","Type":"ContainerDied","Data":"7d030940dac13594c6ec234217429a9145b55a445ab7b81934e33a21c7e23f93"}
Sep 07 16:25:20.805916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.805671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw" event={"ID":"6ab701f6-6926-4d46-a19b-5297fc9babcc","Type":"ContainerStarted","Data":"7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994"}
Sep 07 16:25:20.807393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.807369    2579 generic.go:356] "Generic (PLEG): container finished" podID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerID="2c246baaec83d93c4e06bd2af159fdca4422a6c1fa82fa973236adf734369f8a" exitCode=0
Sep 07 16:25:20.807505 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.807455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g" event={"ID":"a11014ce-b72e-479b-b461-2a59d6be9bb1","Type":"ContainerDied","Data":"2c246baaec83d93c4e06bd2af159fdca4422a6c1fa82fa973236adf734369f8a"}
Sep 07 16:25:20.809387 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.809364    2579 generic.go:356] "Generic (PLEG): container finished" podID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerID="ac34431b41644db44296eb84c249b652b7af8a5f53bcdde74bb802e18912ca2d" exitCode=0
Sep 07 16:25:20.809482 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.809411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl" event={"ID":"a7e78264-6c59-4e5d-9983-ca8b08eb27cc","Type":"ContainerDied","Data":"ac34431b41644db44296eb84c249b652b7af8a5f53bcdde74bb802e18912ca2d"}
Sep 07 16:25:20.809482 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:20.809445    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl" event={"ID":"a7e78264-6c59-4e5d-9983-ca8b08eb27cc","Type":"ContainerStarted","Data":"786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0"}
Sep 07 16:25:21.815941 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.815913    2579 generic.go:356] "Generic (PLEG): container finished" podID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerID="b33c80fefa9dadedbf764780816cf32b283c70bedf2b1f55391d7ee3d61a3cd8" exitCode=0
Sep 07 16:25:21.816299 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.815990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl" event={"ID":"a7e78264-6c59-4e5d-9983-ca8b08eb27cc","Type":"ContainerDied","Data":"b33c80fefa9dadedbf764780816cf32b283c70bedf2b1f55391d7ee3d61a3cd8"}
Sep 07 16:25:21.817922 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.817898    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerID="f9af5625979aa07096ed8eae57b03b000d50f3443183c683d57893e15b2d62e3" exitCode=0
Sep 07 16:25:21.818199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.818017    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6" event={"ID":"9d8749fe-f7a8-4dab-b969-469eb428bad0","Type":"ContainerDied","Data":"f9af5625979aa07096ed8eae57b03b000d50f3443183c683d57893e15b2d62e3"}
Sep 07 16:25:21.820094 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.820074    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerID="29ee4eb31548669f8b19670cadc622af364ea821e02c62bd8778ab90dca24254" exitCode=0
Sep 07 16:25:21.820202 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:21.820094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw" event={"ID":"6ab701f6-6926-4d46-a19b-5297fc9babcc","Type":"ContainerDied","Data":"29ee4eb31548669f8b19670cadc622af364ea821e02c62bd8778ab90dca24254"}
Sep 07 16:25:22.827319 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.827282    2579 generic.go:356] "Generic (PLEG): container finished" podID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerID="d989d35944e490ef6b64574ad8709a9db4ffb0106d9a006829826bd3bc926398" exitCode=0
Sep 07 16:25:22.827767 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.827367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g" event={"ID":"a11014ce-b72e-479b-b461-2a59d6be9bb1","Type":"ContainerDied","Data":"d989d35944e490ef6b64574ad8709a9db4ffb0106d9a006829826bd3bc926398"}
Sep 07 16:25:22.829594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.829566    2579 generic.go:356] "Generic (PLEG): container finished" podID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerID="87ee6eca9c634b370695c2d9819be5689763abd1705eadf591392bf85556d86f" exitCode=0
Sep 07 16:25:22.829696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.829658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl" event={"ID":"a7e78264-6c59-4e5d-9983-ca8b08eb27cc","Type":"ContainerDied","Data":"87ee6eca9c634b370695c2d9819be5689763abd1705eadf591392bf85556d86f"}
Sep 07 16:25:22.831679 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.831658    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerID="59dc70ca55c6d786d734feb07182a173b0a12c755e0f869b25c04d02b0d984e1" exitCode=0
Sep 07 16:25:22.831790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.831705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6" event={"ID":"9d8749fe-f7a8-4dab-b969-469eb428bad0","Type":"ContainerDied","Data":"59dc70ca55c6d786d734feb07182a173b0a12c755e0f869b25c04d02b0d984e1"}
Sep 07 16:25:22.833926 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.833907    2579 generic.go:356] "Generic (PLEG): container finished" podID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerID="cbe7a5af37f580e10a20e8a5e723bf2dbe07e9f8e18a752dd93264d787ba6ef7" exitCode=0
Sep 07 16:25:22.834016 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:22.833934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw" event={"ID":"6ab701f6-6926-4d46-a19b-5297fc9babcc","Type":"ContainerDied","Data":"cbe7a5af37f580e10a20e8a5e723bf2dbe07e9f8e18a752dd93264d787ba6ef7"}
Sep 07 16:25:23.840972 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:23.840935    2579 generic.go:356] "Generic (PLEG): container finished" podID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerID="7c86c9edf929dc4c8eb915325c0a8f480c8a81b058f0200af4fb875c39358db3" exitCode=0
Sep 07 16:25:23.841408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:23.841027    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g" event={"ID":"a11014ce-b72e-479b-b461-2a59d6be9bb1","Type":"ContainerDied","Data":"7c86c9edf929dc4c8eb915325c0a8f480c8a81b058f0200af4fb875c39358db3"}
Sep 07 16:25:23.985159 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:23.985127    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:24.040380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.040353    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:24.043223 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.043203    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:24.075516 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util\") pod \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") "
Sep 07 16:25:24.075516 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075523    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util\") pod \"6ab701f6-6926-4d46-a19b-5297fc9babcc\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075567    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhblv\" (UniqueName: \"kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv\") pod \"6ab701f6-6926-4d46-a19b-5297fc9babcc\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle\") pod \"6ab701f6-6926-4d46-a19b-5297fc9babcc\" (UID: \"6ab701f6-6926-4d46-a19b-5297fc9babcc\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crkk6\" (UniqueName: \"kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6\") pod \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075695    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle\") pod \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\" (UID: \"a7e78264-6c59-4e5d-9983-ca8b08eb27cc\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle\") pod \"9d8749fe-f7a8-4dab-b969-469eb428bad0\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") "
Sep 07 16:25:24.075821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util\") pod \"9d8749fe-f7a8-4dab-b969-469eb428bad0\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") "
Sep 07 16:25:24.076088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.075824    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sh5f\" (UniqueName: \"kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f\") pod \"9d8749fe-f7a8-4dab-b969-469eb428bad0\" (UID: \"9d8749fe-f7a8-4dab-b969-469eb428bad0\") "
Sep 07 16:25:24.076226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.076202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle" (OuterVolumeSpecName: "bundle") pod "6ab701f6-6926-4d46-a19b-5297fc9babcc" (UID: "6ab701f6-6926-4d46-a19b-5297fc9babcc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.077528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.076863    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle" (OuterVolumeSpecName: "bundle") pod "a7e78264-6c59-4e5d-9983-ca8b08eb27cc" (UID: "a7e78264-6c59-4e5d-9983-ca8b08eb27cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.080445 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.080415    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv" (OuterVolumeSpecName: "kube-api-access-xhblv") pod "6ab701f6-6926-4d46-a19b-5297fc9babcc" (UID: "6ab701f6-6926-4d46-a19b-5297fc9babcc"). InnerVolumeSpecName "kube-api-access-xhblv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:24.082572 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.082547    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle" (OuterVolumeSpecName: "bundle") pod "9d8749fe-f7a8-4dab-b969-469eb428bad0" (UID: "9d8749fe-f7a8-4dab-b969-469eb428bad0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.083419 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.083377    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util" (OuterVolumeSpecName: "util") pod "6ab701f6-6926-4d46-a19b-5297fc9babcc" (UID: "6ab701f6-6926-4d46-a19b-5297fc9babcc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.083766 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.083726    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f" (OuterVolumeSpecName: "kube-api-access-9sh5f") pod "9d8749fe-f7a8-4dab-b969-469eb428bad0" (UID: "9d8749fe-f7a8-4dab-b969-469eb428bad0"). InnerVolumeSpecName "kube-api-access-9sh5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:24.084117 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.084095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6" (OuterVolumeSpecName: "kube-api-access-crkk6") pod "a7e78264-6c59-4e5d-9983-ca8b08eb27cc" (UID: "a7e78264-6c59-4e5d-9983-ca8b08eb27cc"). InnerVolumeSpecName "kube-api-access-crkk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:24.085020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.084998    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util" (OuterVolumeSpecName: "util") pod "a7e78264-6c59-4e5d-9983-ca8b08eb27cc" (UID: "a7e78264-6c59-4e5d-9983-ca8b08eb27cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.085585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.085569    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util" (OuterVolumeSpecName: "util") pod "9d8749fe-f7a8-4dab-b969-469eb428bad0" (UID: "9d8749fe-f7a8-4dab-b969-469eb428bad0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177187    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177217    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crkk6\" (UniqueName: \"kubernetes.io/projected/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-kube-api-access-crkk6\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177228    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177236    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177243    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d8749fe-f7a8-4dab-b969-469eb428bad0-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177251    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sh5f\" (UniqueName: \"kubernetes.io/projected/9d8749fe-f7a8-4dab-b969-469eb428bad0-kube-api-access-9sh5f\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177258    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7e78264-6c59-4e5d-9983-ca8b08eb27cc-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177265    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab701f6-6926-4d46-a19b-5297fc9babcc-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.177276 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.177273    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhblv\" (UniqueName: \"kubernetes.io/projected/6ab701f6-6926-4d46-a19b-5297fc9babcc-kube-api-access-xhblv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:24.848271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.848240    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6"
Sep 07 16:25:24.848271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.848251    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053wsjm6" event={"ID":"9d8749fe-f7a8-4dab-b969-469eb428bad0","Type":"ContainerDied","Data":"e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546"}
Sep 07 16:25:24.848857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.848299    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e630c10ac9c8e4bef51921835c4b410a4b9dbe4e91729df401838a537777d546"
Sep 07 16:25:24.850251 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.850226    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw"
Sep 07 16:25:24.850397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.850253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0ksrdw" event={"ID":"6ab701f6-6926-4d46-a19b-5297fc9babcc","Type":"ContainerDied","Data":"7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994"}
Sep 07 16:25:24.850397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.850293    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dbdfb56500f35752981093d462f1f48d6933e4b95a20bce7581d273ccd50994"
Sep 07 16:25:24.852155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.852133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl" event={"ID":"a7e78264-6c59-4e5d-9983-ca8b08eb27cc","Type":"ContainerDied","Data":"786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0"}
Sep 07 16:25:24.852252 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.852173    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="786413783a1cbfbfde5637d0da95a58613ce2593e7e1d05da7226e2456f3a3e0"
Sep 07 16:25:24.852316 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.852304    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c457nl"
Sep 07 16:25:24.979428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.979405    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:24.982760 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.982726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle\") pod \"a11014ce-b72e-479b-b461-2a59d6be9bb1\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") "
Sep 07 16:25:24.982839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.982799    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util\") pod \"a11014ce-b72e-479b-b461-2a59d6be9bb1\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") "
Sep 07 16:25:24.982889 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.982847    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nxqm\" (UniqueName: \"kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm\") pod \"a11014ce-b72e-479b-b461-2a59d6be9bb1\" (UID: \"a11014ce-b72e-479b-b461-2a59d6be9bb1\") "
Sep 07 16:25:24.983458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.983170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle" (OuterVolumeSpecName: "bundle") pod "a11014ce-b72e-479b-b461-2a59d6be9bb1" (UID: "a11014ce-b72e-479b-b461-2a59d6be9bb1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:24.985397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.985378    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm" (OuterVolumeSpecName: "kube-api-access-9nxqm") pod "a11014ce-b72e-479b-b461-2a59d6be9bb1" (UID: "a11014ce-b72e-479b-b461-2a59d6be9bb1"). InnerVolumeSpecName "kube-api-access-9nxqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:24.988921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:24.988893    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util" (OuterVolumeSpecName: "util") pod "a11014ce-b72e-479b-b461-2a59d6be9bb1" (UID: "a11014ce-b72e-479b-b461-2a59d6be9bb1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:25.083624 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.083588    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nxqm\" (UniqueName: \"kubernetes.io/projected/a11014ce-b72e-479b-b461-2a59d6be9bb1-kube-api-access-9nxqm\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:25.083624 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.083616    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:25.083624 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.083627    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a11014ce-b72e-479b-b461-2a59d6be9bb1-util\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:25.859847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.859815    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g"
Sep 07 16:25:25.859847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.859838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbkgr7g" event={"ID":"a11014ce-b72e-479b-b461-2a59d6be9bb1","Type":"ContainerDied","Data":"419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5"}
Sep 07 16:25:25.860261 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:25.859886    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="419134d2527eb844ac472533f6c2e44e270cdcc713ceb3f53be35a7f59104bc5"
Sep 07 16:25:30.588671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.588629    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"]
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589121    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="util"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589139    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="util"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589150    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="pull"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589158    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="pull"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589168    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="util"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589179    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="util"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589189    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="pull"
Sep 07 16:25:30.589199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589196    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="pull"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589204    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589213    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589225    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="pull"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589234    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="pull"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589244    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="util"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589253    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="util"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589267    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589275    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589283    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589291    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589308    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589316    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589326    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="util"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589333    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="util"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589342    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="pull"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589349    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="pull"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589439    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8749fe-f7a8-4dab-b969-469eb428bad0" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589456    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e78264-6c59-4e5d-9983-ca8b08eb27cc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589468    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ab701f6-6926-4d46-a19b-5297fc9babcc" containerName="extract"
Sep 07 16:25:30.589570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.589476    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a11014ce-b72e-479b-b461-2a59d6be9bb1" containerName="extract"
Sep 07 16:25:30.592978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.592957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:30.595808 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.595782    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xxvhp\""
Sep 07 16:25:30.595912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.595839    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 16:25:30.595912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.595786    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 16:25:30.595912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.595905    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 07 16:25:30.604154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.604130    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"]
Sep 07 16:25:30.627160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.627125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54j9\" (UniqueName: \"kubernetes.io/projected/32482f3f-fc42-4cd4-bbda-e20c251a56ca-kube-api-access-p54j9\") pod \"dns-operator-controller-manager-5985564776-xvkd2\" (UID: \"32482f3f-fc42-4cd4-bbda-e20c251a56ca\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:30.728038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.727998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54j9\" (UniqueName: \"kubernetes.io/projected/32482f3f-fc42-4cd4-bbda-e20c251a56ca-kube-api-access-p54j9\") pod \"dns-operator-controller-manager-5985564776-xvkd2\" (UID: \"32482f3f-fc42-4cd4-bbda-e20c251a56ca\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:30.738372 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.738338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54j9\" (UniqueName: \"kubernetes.io/projected/32482f3f-fc42-4cd4-bbda-e20c251a56ca-kube-api-access-p54j9\") pod \"dns-operator-controller-manager-5985564776-xvkd2\" (UID: \"32482f3f-fc42-4cd4-bbda-e20c251a56ca\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:30.903961 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:30.903860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:31.176189 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:31.176162    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"]
Sep 07 16:25:31.177771 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:31.177721    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32482f3f_fc42_4cd4_bbda_e20c251a56ca.slice/crio-410af93d0c8f8ee2055bdaa56cb10e584cd86b30cdb2731f164ed0efc0aa8fe8 WatchSource:0}: Error finding container 410af93d0c8f8ee2055bdaa56cb10e584cd86b30cdb2731f164ed0efc0aa8fe8: Status 404 returned error can't find the container with id 410af93d0c8f8ee2055bdaa56cb10e584cd86b30cdb2731f164ed0efc0aa8fe8
Sep 07 16:25:31.893910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:31.893825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2" event={"ID":"32482f3f-fc42-4cd4-bbda-e20c251a56ca","Type":"ContainerStarted","Data":"410af93d0c8f8ee2055bdaa56cb10e584cd86b30cdb2731f164ed0efc0aa8fe8"}
Sep 07 16:25:32.586996 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.586956    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"]
Sep 07 16:25:32.590577 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.590556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:32.592956 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.592929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-29jlw\""
Sep 07 16:25:32.599226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.599203    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"]
Sep 07 16:25:32.646645 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.646603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzmjf\" (UniqueName: \"kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf\") pod \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" (UID: \"ab0c6561-4452-4cf5-8bd0-e28f141d1174\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:32.747949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.747916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzmjf\" (UniqueName: \"kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf\") pod \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" (UID: \"ab0c6561-4452-4cf5-8bd0-e28f141d1174\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:32.757615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.757531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzmjf\" (UniqueName: \"kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf\") pod \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" (UID: \"ab0c6561-4452-4cf5-8bd0-e28f141d1174\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:32.902669 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:32.902590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:33.861596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:33.861515    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"]
Sep 07 16:25:33.865268 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:33.865243    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0c6561_4452_4cf5_8bd0_e28f141d1174.slice/crio-c1030ed7894801f77e54cacba6532cec7ea601f85c9bb7c65a46a621ac7ad092 WatchSource:0}: Error finding container c1030ed7894801f77e54cacba6532cec7ea601f85c9bb7c65a46a621ac7ad092: Status 404 returned error can't find the container with id c1030ed7894801f77e54cacba6532cec7ea601f85c9bb7c65a46a621ac7ad092
Sep 07 16:25:33.905821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:33.905788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2" event={"ID":"32482f3f-fc42-4cd4-bbda-e20c251a56ca","Type":"ContainerStarted","Data":"e3a6bc056c42b8089df7c5d26cb0a20b4ff6380093484d7d13fe09188871a8c8"}
Sep 07 16:25:33.906207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:33.905960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:33.907224 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:33.907199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8" event={"ID":"ab0c6561-4452-4cf5-8bd0-e28f141d1174","Type":"ContainerStarted","Data":"c1030ed7894801f77e54cacba6532cec7ea601f85c9bb7c65a46a621ac7ad092"}
Sep 07 16:25:33.927209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:33.927168    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2" podStartSLOduration=1.443596677 podStartE2EDuration="3.927155656s" totalImagesPullingTime="2.483558979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:33.924430403 +0000 UTC m=+372.568725734" watchObservedRunningTime="2026-09-07 16:25:33.927155656 +0000 UTC m=+372.571450964"
Sep 07 16:25:36.925430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:36.925396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8" event={"ID":"ab0c6561-4452-4cf5-8bd0-e28f141d1174","Type":"ContainerStarted","Data":"70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04"}
Sep 07 16:25:36.925825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:36.925548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:36.958833 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:36.958789    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8" podStartSLOduration=2.959499239 podStartE2EDuration="4.958774769s" totalImagesPullingTime="1.99927553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:36.956558954 +0000 UTC m=+375.600854284" watchObservedRunningTime="2026-09-07 16:25:36.958774769 +0000 UTC m=+375.603070119"
Sep 07 16:25:37.107509 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.107474    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"]
Sep 07 16:25:37.111106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.111076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.113334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.113309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z2dkf\""
Sep 07 16:25:37.123340 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.123317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"]
Sep 07 16:25:37.186301 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.186217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.186301 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.186256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrmlp\" (UniqueName: \"kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.286996 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.286958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.287187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.287004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrmlp\" (UniqueName: \"kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.287346 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.287323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.296728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.296704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrmlp\" (UniqueName: \"kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp\") pod \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.424167 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.424131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:37.701103 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.701073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"]
Sep 07 16:25:37.703911 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:37.703877    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7443437a_adc4_410b_b0e5_387b1d5c5947.slice/crio-59dc8fcf5e0e2b9608dbf89d6754eb4264d923edfddfc9e6665988047e572546 WatchSource:0}: Error finding container 59dc8fcf5e0e2b9608dbf89d6754eb4264d923edfddfc9e6665988047e572546: Status 404 returned error can't find the container with id 59dc8fcf5e0e2b9608dbf89d6754eb4264d923edfddfc9e6665988047e572546
Sep 07 16:25:37.932541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:37.932452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh" event={"ID":"7443437a-adc4-410b-b0e5-387b1d5c5947","Type":"ContainerStarted","Data":"59dc8fcf5e0e2b9608dbf89d6754eb4264d923edfddfc9e6665988047e572546"}
Sep 07 16:25:39.619612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.619580    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-wm5ss"]
Sep 07 16:25:39.623371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.623352    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:39.626096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.626077    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7lq26\""
Sep 07 16:25:39.632887 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.632864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-wm5ss"]
Sep 07 16:25:39.711691 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.711656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd2qp\" (UniqueName: \"kubernetes.io/projected/d200e072-bd02-4e20-9681-8f69e53ed62c-kube-api-access-nd2qp\") pod \"authorino-operator-68d88948d8-wm5ss\" (UID: \"d200e072-bd02-4e20-9681-8f69e53ed62c\") " pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:39.812954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.812912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd2qp\" (UniqueName: \"kubernetes.io/projected/d200e072-bd02-4e20-9681-8f69e53ed62c-kube-api-access-nd2qp\") pod \"authorino-operator-68d88948d8-wm5ss\" (UID: \"d200e072-bd02-4e20-9681-8f69e53ed62c\") " pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:39.823701 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.823653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd2qp\" (UniqueName: \"kubernetes.io/projected/d200e072-bd02-4e20-9681-8f69e53ed62c-kube-api-access-nd2qp\") pod \"authorino-operator-68d88948d8-wm5ss\" (UID: \"d200e072-bd02-4e20-9681-8f69e53ed62c\") " pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:39.936489 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:39.936411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:40.214454 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:40.214426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-wm5ss"]
Sep 07 16:25:41.177403 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:41.177357    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd200e072_bd02_4e20_9681_8f69e53ed62c.slice/crio-f8f2d09f71d9cf4beba4ced731066b52af6ba836fa83d6932c200b76a91c3148 WatchSource:0}: Error finding container f8f2d09f71d9cf4beba4ced731066b52af6ba836fa83d6932c200b76a91c3148: Status 404 returned error can't find the container with id f8f2d09f71d9cf4beba4ced731066b52af6ba836fa83d6932c200b76a91c3148
Sep 07 16:25:41.959865 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:41.959823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh" event={"ID":"7443437a-adc4-410b-b0e5-387b1d5c5947","Type":"ContainerStarted","Data":"8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093"}
Sep 07 16:25:41.960070 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:41.960011    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:41.961173 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:41.961150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss" event={"ID":"d200e072-bd02-4e20-9681-8f69e53ed62c","Type":"ContainerStarted","Data":"f8f2d09f71d9cf4beba4ced731066b52af6ba836fa83d6932c200b76a91c3148"}
Sep 07 16:25:41.979446 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:41.979385    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh" podStartSLOduration=1.4388303740000001 podStartE2EDuration="4.979366466s" totalImagesPullingTime="3.540536092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:41.977174017 +0000 UTC m=+380.621469344" watchObservedRunningTime="2026-09-07 16:25:41.979366466 +0000 UTC m=+380.623661777"
Sep 07 16:25:42.911444 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:42.911404    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-575ccb6ffc-gzkfk"]
Sep 07 16:25:42.915080 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:42.915060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:42.920506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:42.920449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575ccb6ffc-gzkfk"]
Sep 07 16:25:43.042650 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.042650 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-trusted-ca-bundle\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.042933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.042933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-oauth-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.042933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-service-ca\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.043086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-oauth-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.043086 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.042985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxr7m\" (UniqueName: \"kubernetes.io/projected/71f40916-9b6f-407a-86a4-1a798a37c1ef-kube-api-access-zxr7m\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.143618 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-oauth-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.143829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-service-ca\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.143829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-oauth-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.143829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxr7m\" (UniqueName: \"kubernetes.io/projected/71f40916-9b6f-407a-86a4-1a798a37c1ef-kube-api-access-zxr7m\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.143829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.144107 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-trusted-ca-bundle\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.144107 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.143922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.144558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.144507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-oauth-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.144691 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.144670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-service-ca\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.144740 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.144670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.145015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.144996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71f40916-9b6f-407a-86a4-1a798a37c1ef-trusted-ca-bundle\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.146279 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.146261    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-oauth-config\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.146387 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.146368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71f40916-9b6f-407a-86a4-1a798a37c1ef-console-serving-cert\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.151050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.151028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxr7m\" (UniqueName: \"kubernetes.io/projected/71f40916-9b6f-407a-86a4-1a798a37c1ef-kube-api-access-zxr7m\") pod \"console-575ccb6ffc-gzkfk\" (UID: \"71f40916-9b6f-407a-86a4-1a798a37c1ef\") " pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.227145 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.227116    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:43.504551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.504516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575ccb6ffc-gzkfk"]
Sep 07 16:25:43.506690 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:43.506666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f40916_9b6f_407a_86a4_1a798a37c1ef.slice/crio-129251af04cb13f540a9773c24eb8d1fbcfcbfd1efa0409267b6f8d150d60673 WatchSource:0}: Error finding container 129251af04cb13f540a9773c24eb8d1fbcfcbfd1efa0409267b6f8d150d60673: Status 404 returned error can't find the container with id 129251af04cb13f540a9773c24eb8d1fbcfcbfd1efa0409267b6f8d150d60673
Sep 07 16:25:43.974775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.974715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss" event={"ID":"d200e072-bd02-4e20-9681-8f69e53ed62c","Type":"ContainerStarted","Data":"d8982433186c29bba546db364fec4537f92e376bacd592a18ca1f2390d65759b"}
Sep 07 16:25:43.975216 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.974817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:43.976775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.976699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575ccb6ffc-gzkfk" event={"ID":"71f40916-9b6f-407a-86a4-1a798a37c1ef","Type":"ContainerStarted","Data":"5ee4b99dcd9d84d97e149ead0f8cbf73bb20c7b3afb6319d0319861b9a94c94f"}
Sep 07 16:25:43.976775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.976770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575ccb6ffc-gzkfk" event={"ID":"71f40916-9b6f-407a-86a4-1a798a37c1ef","Type":"ContainerStarted","Data":"129251af04cb13f540a9773c24eb8d1fbcfcbfd1efa0409267b6f8d150d60673"}
Sep 07 16:25:43.993196 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:43.993145    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss" podStartSLOduration=3.255969371 podStartE2EDuration="4.993126651s" totalImagesPullingTime="1.73715728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:43.991688663 +0000 UTC m=+382.635984030" watchObservedRunningTime="2026-09-07 16:25:43.993126651 +0000 UTC m=+382.637421960"
Sep 07 16:25:44.016693 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.016660    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"]
Sep 07 16:25:44.018838 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.018792    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-575ccb6ffc-gzkfk" podStartSLOduration=2.018776807 podStartE2EDuration="2.018776807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:44.014482602 +0000 UTC m=+382.658777922" watchObservedRunningTime="2026-09-07 16:25:44.018776807 +0000 UTC m=+382.663072115"
Sep 07 16:25:44.020416 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.020381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.022894 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.022867    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 07 16:25:44.023000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.022954    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vkvzn\""
Sep 07 16:25:44.023000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.022990    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 07 16:25:44.027027 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.027002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"]
Sep 07 16:25:44.152232 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.152189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.152409 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.152240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgh2s\" (UniqueName: \"kubernetes.io/projected/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-kube-api-access-vgh2s\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.152409 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.152350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.253722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.253632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.253891 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.253785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.253891 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.253824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgh2s\" (UniqueName: \"kubernetes.io/projected/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-kube-api-access-vgh2s\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.254228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.254203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.256212 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.256192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.261425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.261402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgh2s\" (UniqueName: \"kubernetes.io/projected/f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd-kube-api-access-vgh2s\") pod \"kuadrant-console-plugin-598f86468b-klrtm\" (UID: \"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.332126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.332096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"
Sep 07 16:25:44.603883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.603855    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm"]
Sep 07 16:25:44.605467 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:44.605444    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4c6ed83_aeb0_4484_9b87_ad80b4aab4bd.slice/crio-85a7c4a97e3432d1809e2491093e0275284033f662822440f588c8dd06caae53 WatchSource:0}: Error finding container 85a7c4a97e3432d1809e2491093e0275284033f662822440f588c8dd06caae53: Status 404 returned error can't find the container with id 85a7c4a97e3432d1809e2491093e0275284033f662822440f588c8dd06caae53
Sep 07 16:25:44.915165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.915086    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-xvkd2"
Sep 07 16:25:44.983963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:44.983924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm" event={"ID":"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd","Type":"ContainerStarted","Data":"85a7c4a97e3432d1809e2491093e0275284033f662822440f588c8dd06caae53"}
Sep 07 16:25:47.935344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:47.935307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:50.016828 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:50.016788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm" event={"ID":"f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd","Type":"ContainerStarted","Data":"8005cd72c79428bd4bc06db5c822ff7c083f505f5409b59a5f2cfda4daadc2bb"}
Sep 07 16:25:50.033128 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:50.033080    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-klrtm" podStartSLOduration=2.409215513 podStartE2EDuration="7.03306466s" totalImagesPullingTime="4.623849147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:50.032003746 +0000 UTC m=+388.676299056" watchObservedRunningTime="2026-09-07 16:25:50.03306466 +0000 UTC m=+388.677359968"
Sep 07 16:25:52.969008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:52.968975    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:53.227878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:53.227781    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:53.227878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:53.227859    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:53.232573 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:53.232549    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:54.041965 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.041927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-575ccb6ffc-gzkfk"
Sep 07 16:25:54.098095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.098055    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:25:54.279781 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.279734    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"]
Sep 07 16:25:54.284569 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.284549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.297951 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.297891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"]
Sep 07 16:25:54.346178 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.346135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.346178 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.346179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f8zf\" (UniqueName: \"kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.447734 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.447692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.447944 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.447736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f8zf\" (UniqueName: \"kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.448122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.448099    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.455633 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.455607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f8zf\" (UniqueName: \"kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.594810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.594689    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:54.873358 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.873331    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"]
Sep 07 16:25:54.875172 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:54.875138    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0944a5_7621_4266_af8a_5cd3a7f58cff.slice/crio-38e375496e10d33430aa41d39f989d9b7b469cfd6b1d398fece3be923db93874 WatchSource:0}: Error finding container 38e375496e10d33430aa41d39f989d9b7b469cfd6b1d398fece3be923db93874: Status 404 returned error can't find the container with id 38e375496e10d33430aa41d39f989d9b7b469cfd6b1d398fece3be923db93874
Sep 07 16:25:54.941382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.941355    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"]
Sep 07 16:25:54.941547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.941527    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" containerName="manager" containerID="cri-o://8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093" gracePeriod=2
Sep 07 16:25:54.957711 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.957681    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"]
Sep 07 16:25:54.964725 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:54.964678    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7443437a-adc4-410b-b0e5-387b1d5c5947\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-w54rh\": pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:54.965541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.965502    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"]
Sep 07 16:25:54.968820 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.968782    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:25:54.969355 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.969336    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" containerName="manager"
Sep 07 16:25:54.969355 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.969356    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" containerName="manager"
Sep 07 16:25:54.969559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.969547    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" containerName="manager"
Sep 07 16:25:54.973271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.973251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:54.974225 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.974203    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"]
Sep 07 16:25:54.975859 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:54.975816    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:54.986350 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.986290    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-wm5ss"
Sep 07 16:25:54.988512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.988475    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:25:54.998120 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.998090    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:25:54.998627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.998612    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" containerName="manager"
Sep 07 16:25:54.998704 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.998631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" containerName="manager"
Sep 07 16:25:54.998784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:54.998764    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" containerName="manager"
Sep 07 16:25:55.002466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.002446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.011565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.011537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:25:55.018356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.018325    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"]
Sep 07 16:25:55.018630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.018577    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" containerName="manager" containerID="cri-o://70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04" gracePeriod=2
Sep 07 16:25:55.028716 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.028675    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.029208 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.029191    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"]
Sep 07 16:25:55.030545 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.030518    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.045381 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.045346    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" containerName="manager" containerID="cri-o://4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee" gracePeriod=2
Sep 07 16:25:55.056979 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.056941    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.058311 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.058278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94t4f\" (UniqueName: \"kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.058457 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.058406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.058705 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.058682    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.060238 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.060212    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:55.064061 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.064032    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.159896 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.159854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94t4f\" (UniqueName: \"kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.160197 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.160090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.160197 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.160163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr7bv\" (UniqueName: \"kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.160305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.160284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.161088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.160649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.168253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.168194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94t4f\" (UniqueName: \"kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f\") pod \"kuadrant-operator-controller-manager-58fc56757c-4nf8p\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.250376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.250349    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.252535 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.252507    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.254213 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.254188    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:55.255739 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.255719    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.261148 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.261127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.261238 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.261162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr7bv\" (UniqueName: \"kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.261533 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.261512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.285335 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.285301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr7bv\" (UniqueName: \"kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tcmpb\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.288678 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.288659    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.291854 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.291832    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:55.292134 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.292109    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:55.294182 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.294158    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.297695 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.297672    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.300303 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.300279    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-w54rh\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:55.302175 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.302156    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:55.304068 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:55.304046    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-vtcm8\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:55.321310 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.321268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:55.332304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.332276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:25:55.363058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f8zf\" (UniqueName: \"kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf\") pod \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") "
Sep 07 16:25:55.363254 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363158    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume\") pod \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\" (UID: \"bf0944a5-7621-4266-af8a-5cd3a7f58cff\") "
Sep 07 16:25:55.363254 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363193    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume\") pod \"7443437a-adc4-410b-b0e5-387b1d5c5947\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") "
Sep 07 16:25:55.363254 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzmjf\" (UniqueName: \"kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf\") pod \"ab0c6561-4452-4cf5-8bd0-e28f141d1174\" (UID: \"ab0c6561-4452-4cf5-8bd0-e28f141d1174\") "
Sep 07 16:25:55.363582 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363482    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf0944a5-7621-4266-af8a-5cd3a7f58cff" (UID: "bf0944a5-7621-4266-af8a-5cd3a7f58cff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:55.363674 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363609    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrmlp\" (UniqueName: \"kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp\") pod \"7443437a-adc4-410b-b0e5-387b1d5c5947\" (UID: \"7443437a-adc4-410b-b0e5-387b1d5c5947\") "
Sep 07 16:25:55.363811 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363788    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7443437a-adc4-410b-b0e5-387b1d5c5947" (UID: "7443437a-adc4-410b-b0e5-387b1d5c5947"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:25:55.364012 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.363992    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf0944a5-7621-4266-af8a-5cd3a7f58cff-extensions-socket-volume\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:55.364082 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.364018    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7443437a-adc4-410b-b0e5-387b1d5c5947-extensions-socket-volume\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:55.365721 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.365689    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf" (OuterVolumeSpecName: "kube-api-access-gzmjf") pod "ab0c6561-4452-4cf5-8bd0-e28f141d1174" (UID: "ab0c6561-4452-4cf5-8bd0-e28f141d1174"). InnerVolumeSpecName "kube-api-access-gzmjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:55.366113 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.366086    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp" (OuterVolumeSpecName: "kube-api-access-nrmlp") pod "7443437a-adc4-410b-b0e5-387b1d5c5947" (UID: "7443437a-adc4-410b-b0e5-387b1d5c5947"). InnerVolumeSpecName "kube-api-access-nrmlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:55.366376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.366357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf" (OuterVolumeSpecName: "kube-api-access-8f8zf") pod "bf0944a5-7621-4266-af8a-5cd3a7f58cff" (UID: "bf0944a5-7621-4266-af8a-5cd3a7f58cff"). InnerVolumeSpecName "kube-api-access-8f8zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:25:55.465380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.465324    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzmjf\" (UniqueName: \"kubernetes.io/projected/ab0c6561-4452-4cf5-8bd0-e28f141d1174-kube-api-access-gzmjf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:55.465380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.465358    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrmlp\" (UniqueName: \"kubernetes.io/projected/7443437a-adc4-410b-b0e5-387b1d5c5947-kube-api-access-nrmlp\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:55.465380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.465374    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f8zf\" (UniqueName: \"kubernetes.io/projected/bf0944a5-7621-4266-af8a-5cd3a7f58cff-kube-api-access-8f8zf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:25:55.611911 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.611877    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:25:55.615460 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:55.615435    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb213783e_5ded_4846_8f80_a455166a3320.slice/crio-a0ee841bebd79c93971a94d05a3903c17ac6054548ea5af2cbccbe3ed2005a7c WatchSource:0}: Error finding container a0ee841bebd79c93971a94d05a3903c17ac6054548ea5af2cbccbe3ed2005a7c: Status 404 returned error can't find the container with id a0ee841bebd79c93971a94d05a3903c17ac6054548ea5af2cbccbe3ed2005a7c
Sep 07 16:25:55.623639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.623611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:25:55.624297 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:25:55.624267    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541645af_807f_4310_b0fe_ee5f481c39e7.slice/crio-deaaaf89d9a15aee3d24c202ef20656b607c73dca495e71424d3b42e88be5f3c WatchSource:0}: Error finding container deaaaf89d9a15aee3d24c202ef20656b607c73dca495e71424d3b42e88be5f3c: Status 404 returned error can't find the container with id deaaaf89d9a15aee3d24c202ef20656b607c73dca495e71424d3b42e88be5f3c
Sep 07 16:25:55.888442 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.888367    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7443437a-adc4-410b-b0e5-387b1d5c5947" path="/var/lib/kubelet/pods/7443437a-adc4-410b-b0e5-387b1d5c5947/volumes"
Sep 07 16:25:55.888798 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.888782    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" path="/var/lib/kubelet/pods/ab0c6561-4452-4cf5-8bd0-e28f141d1174/volumes"
Sep 07 16:25:55.889117 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:55.889103    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" path="/var/lib/kubelet/pods/bf0944a5-7621-4266-af8a-5cd3a7f58cff/volumes"
Sep 07 16:25:56.052162 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.052126    2579 generic.go:356] "Generic (PLEG): container finished" podID="7443437a-adc4-410b-b0e5-387b1d5c5947" containerID="8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093" exitCode=0
Sep 07 16:25:56.052597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.052178    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-w54rh"
Sep 07 16:25:56.052597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.052232    2579 scope.go:117] "RemoveContainer" containerID="8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093"
Sep 07 16:25:56.053884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.053856    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" containerID="4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee" exitCode=2
Sep 07 16:25:56.054011 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.053935    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:56.056047 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.056009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" event={"ID":"b213783e-5ded-4846-8f80-a455166a3320","Type":"ContainerStarted","Data":"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"}
Sep 07 16:25:56.056166 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.056104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" event={"ID":"b213783e-5ded-4846-8f80-a455166a3320","Type":"ContainerStarted","Data":"a0ee841bebd79c93971a94d05a3903c17ac6054548ea5af2cbccbe3ed2005a7c"}
Sep 07 16:25:56.056273 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.056258    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:25:56.057309 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.057288    2579 generic.go:356] "Generic (PLEG): container finished" podID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" containerID="70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04" exitCode=0
Sep 07 16:25:56.057404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.057329    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtcm8"
Sep 07 16:25:56.059442 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.059416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" event={"ID":"541645af-807f-4310-b0fe-ee5f481c39e7","Type":"ContainerStarted","Data":"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"}
Sep 07 16:25:56.059549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.059466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" event={"ID":"541645af-807f-4310-b0fe-ee5f481c39e7","Type":"ContainerStarted","Data":"deaaaf89d9a15aee3d24c202ef20656b607c73dca495e71424d3b42e88be5f3c"}
Sep 07 16:25:56.062798 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.062781    2579 scope.go:117] "RemoveContainer" containerID="8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093"
Sep 07 16:25:56.063071 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:56.063053    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093\": container with ID starting with 8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093 not found: ID does not exist" containerID="8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093"
Sep 07 16:25:56.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.063078    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093"} err="failed to get container status \"8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093\": rpc error: code = NotFound desc = could not find container \"8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093\": container with ID starting with 8f589268e38df2b28c5820539791b9b0fc6a7d596a50eb22affa6511698b6093 not found: ID does not exist"
Sep 07 16:25:56.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.063101    2579 scope.go:117] "RemoveContainer" containerID="4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee"
Sep 07 16:25:56.071446 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.071428    2579 scope.go:117] "RemoveContainer" containerID="4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee"
Sep 07 16:25:56.071729 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:56.071705    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee\": container with ID starting with 4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee not found: ID does not exist" containerID="4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee"
Sep 07 16:25:56.071840 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.071733    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee"} err="failed to get container status \"4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee\": rpc error: code = NotFound desc = could not find container \"4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee\": container with ID starting with 4aba227478710b01a7e861e6b1c0e2378602390fe2cf80633a99f2583e8f55ee not found: ID does not exist"
Sep 07 16:25:56.071840 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.071770    2579 scope.go:117] "RemoveContainer" containerID="70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04"
Sep 07 16:25:56.074998 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.074954    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" podStartSLOduration=2.074941605 podStartE2EDuration="2.074941605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:56.073060993 +0000 UTC m=+394.717356301" watchObservedRunningTime="2026-09-07 16:25:56.074941605 +0000 UTC m=+394.719236913"
Sep 07 16:25:56.083247 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.083222    2579 scope.go:117] "RemoveContainer" containerID="70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04"
Sep 07 16:25:56.083683 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:56.083660    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04\": container with ID starting with 70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04 not found: ID does not exist" containerID="70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04"
Sep 07 16:25:56.083796 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.083711    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04"} err="failed to get container status \"70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04\": rpc error: code = NotFound desc = could not find container \"70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04\": container with ID starting with 70a559426512c799aacf5c6fab7276db920b8c551f33111d101a6b436f497f04 not found: ID does not exist"
Sep 07 16:25:56.090958 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:25:56.090931    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-jj6kv\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="bf0944a5-7621-4266-af8a-5cd3a7f58cff" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jj6kv"
Sep 07 16:25:56.091615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:56.091577    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" podStartSLOduration=2.091564897 podStartE2EDuration="2.091564897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:25:56.089167429 +0000 UTC m=+394.733462782" watchObservedRunningTime="2026-09-07 16:25:56.091564897 +0000 UTC m=+394.735860206"
Sep 07 16:25:57.068199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:25:57.068166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:26:07.070087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:07.070055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:26:08.076044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.076013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:26:08.129040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.129004    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:26:08.129287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.129256    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" podUID="b213783e-5ded-4846-8f80-a455166a3320" containerName="manager" containerID="cri-o://d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8" gracePeriod=10
Sep 07 16:26:08.386611 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.386584    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:26:08.386758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.386583    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:26:08.387079 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387066    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b213783e-5ded-4846-8f80-a455166a3320" containerName="manager"
Sep 07 16:26:08.387134 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387081    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b213783e-5ded-4846-8f80-a455166a3320" containerName="manager"
Sep 07 16:26:08.387134 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387105    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" containerName="manager"
Sep 07 16:26:08.387134 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387113    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" containerName="manager"
Sep 07 16:26:08.387226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387193    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b213783e-5ded-4846-8f80-a455166a3320" containerName="manager"
Sep 07 16:26:08.387226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.387210    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab0c6561-4452-4cf5-8bd0-e28f141d1174" containerName="manager"
Sep 07 16:26:08.390043 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.390022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.405705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.405673    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:26:08.582590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.582557    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume\") pod \"b213783e-5ded-4846-8f80-a455166a3320\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") "
Sep 07 16:26:08.582812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.582628    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94t4f\" (UniqueName: \"kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f\") pod \"b213783e-5ded-4846-8f80-a455166a3320\" (UID: \"b213783e-5ded-4846-8f80-a455166a3320\") "
Sep 07 16:26:08.582812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.582739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfj5\" (UniqueName: \"kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.582956 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.582881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.583062 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.583038    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b213783e-5ded-4846-8f80-a455166a3320" (UID: "b213783e-5ded-4846-8f80-a455166a3320"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:26:08.584896 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.584872    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f" (OuterVolumeSpecName: "kube-api-access-94t4f") pod "b213783e-5ded-4846-8f80-a455166a3320" (UID: "b213783e-5ded-4846-8f80-a455166a3320"). InnerVolumeSpecName "kube-api-access-94t4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:26:08.684018 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.683924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.684018 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.683977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfj5\" (UniqueName: \"kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.684201 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.684034    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b213783e-5ded-4846-8f80-a455166a3320-extensions-socket-volume\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:08.684201 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.684046    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94t4f\" (UniqueName: \"kubernetes.io/projected/b213783e-5ded-4846-8f80-a455166a3320-kube-api-access-94t4f\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:08.684307 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.684285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.694723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.694690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfj5\" (UniqueName: \"kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-cd64g\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.700456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.700426    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:08.976170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:08.976142    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:26:08.977911 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:26:08.977885    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf76e0d4_3f97_4f8c_95aa_d48f24d0e60d.slice/crio-420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5 WatchSource:0}: Error finding container 420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5: Status 404 returned error can't find the container with id 420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5
Sep 07 16:26:09.142738 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.142699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" event={"ID":"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d","Type":"ContainerStarted","Data":"b5920366b24f3cd0d97599d9c92209340fe780a5e7368acb66bdfa5fa5c2da36"}
Sep 07 16:26:09.143184 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.142810    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:09.143184 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.142842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" event={"ID":"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d","Type":"ContainerStarted","Data":"420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5"}
Sep 07 16:26:09.144325 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.144296    2579 generic.go:356] "Generic (PLEG): container finished" podID="b213783e-5ded-4846-8f80-a455166a3320" containerID="d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8" exitCode=0
Sep 07 16:26:09.144458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.144338    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"
Sep 07 16:26:09.144458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.144369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" event={"ID":"b213783e-5ded-4846-8f80-a455166a3320","Type":"ContainerDied","Data":"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"}
Sep 07 16:26:09.144458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.144418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p" event={"ID":"b213783e-5ded-4846-8f80-a455166a3320","Type":"ContainerDied","Data":"a0ee841bebd79c93971a94d05a3903c17ac6054548ea5af2cbccbe3ed2005a7c"}
Sep 07 16:26:09.144458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.144442    2579 scope.go:117] "RemoveContainer" containerID="d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"
Sep 07 16:26:09.159271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.159242    2579 scope.go:117] "RemoveContainer" containerID="d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"
Sep 07 16:26:09.159579 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:26:09.159561    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8\": container with ID starting with d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8 not found: ID does not exist" containerID="d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"
Sep 07 16:26:09.159637 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.159588    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8"} err="failed to get container status \"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8\": rpc error: code = NotFound desc = could not find container \"d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8\": container with ID starting with d333487a4448e5a41e909823db25f2219c96c7fe1741b51e60a6ebd49376dcb8 not found: ID does not exist"
Sep 07 16:26:09.165845 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.165802    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" podStartSLOduration=1.16578607 podStartE2EDuration="1.16578607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:26:09.164929898 +0000 UTC m=+407.809225243" watchObservedRunningTime="2026-09-07 16:26:09.16578607 +0000 UTC m=+407.810081379"
Sep 07 16:26:09.180662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.180632    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:26:09.188270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.188241    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4nf8p"]
Sep 07 16:26:09.889824 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:09.889787    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b213783e-5ded-4846-8f80-a455166a3320" path="/var/lib/kubelet/pods/b213783e-5ded-4846-8f80-a455166a3320/volumes"
Sep 07 16:26:19.123478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.123403    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f9887c959-778zm" podUID="0280167b-4630-4e08-baf8-cc1779249294" containerName="console" containerID="cri-o://a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982" gracePeriod=15
Sep 07 16:26:19.380517 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.380451    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:26:19.461692 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461655    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461692 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461696    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461781    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75p4c\" (UniqueName: \"kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.461963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.461881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca\") pod \"0280167b-4630-4e08-baf8-cc1779249294\" (UID: \"0280167b-4630-4e08-baf8-cc1779249294\") "
Sep 07 16:26:19.462210 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.462187    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config" (OuterVolumeSpecName: "console-config") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:26:19.462299 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.462274    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:26:19.462431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.462287    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:26:19.462431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.462371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca" (OuterVolumeSpecName: "service-ca") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:26:19.464406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.464379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:26:19.464503 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.464379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:26:19.464503 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.464446    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c" (OuterVolumeSpecName: "kube-api-access-75p4c") pod "0280167b-4630-4e08-baf8-cc1779249294" (UID: "0280167b-4630-4e08-baf8-cc1779249294"). InnerVolumeSpecName "kube-api-access-75p4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:26:19.562851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562817    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-console-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.562851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562844    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-trusted-ca-bundle\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.562851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562854    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-oauth-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.562851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562864    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-serving-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.562851 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562872    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0280167b-4630-4e08-baf8-cc1779249294-console-oauth-config\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.563185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562882    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75p4c\" (UniqueName: \"kubernetes.io/projected/0280167b-4630-4e08-baf8-cc1779249294-kube-api-access-75p4c\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:19.563185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:19.562891    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0280167b-4630-4e08-baf8-cc1779249294-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:20.154930 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.154845    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:26:20.201824 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.201790    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:26:20.202000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.201979    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" podUID="541645af-807f-4310-b0fe-ee5f481c39e7" containerName="manager" containerID="cri-o://54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f" gracePeriod=10
Sep 07 16:26:20.217615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.217585    2579 generic.go:356] "Generic (PLEG): container finished" podID="0280167b-4630-4e08-baf8-cc1779249294" containerID="a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982" exitCode=0
Sep 07 16:26:20.217790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.217632    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9887c959-778zm"
Sep 07 16:26:20.217790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.217666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9887c959-778zm" event={"ID":"0280167b-4630-4e08-baf8-cc1779249294","Type":"ContainerDied","Data":"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"}
Sep 07 16:26:20.217790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.217729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9887c959-778zm" event={"ID":"0280167b-4630-4e08-baf8-cc1779249294","Type":"ContainerDied","Data":"e0a75ef91c4de7dabd72f3e8da10969a360b331da0fa084c3903cbed102f19fc"}
Sep 07 16:26:20.217980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.217792    2579 scope.go:117] "RemoveContainer" containerID="a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"
Sep 07 16:26:20.229056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.229037    2579 scope.go:117] "RemoveContainer" containerID="a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"
Sep 07 16:26:20.229379 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:26:20.229361    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982\": container with ID starting with a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982 not found: ID does not exist" containerID="a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"
Sep 07 16:26:20.229462 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.229387    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982"} err="failed to get container status \"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982\": rpc error: code = NotFound desc = could not find container \"a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982\": container with ID starting with a0e5bfed724dbeed5ce94050359b8aa29c37030e23ecbe70857358fa4021d982 not found: ID does not exist"
Sep 07 16:26:20.239842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.239816    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:26:20.245927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.245894    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f9887c959-778zm"]
Sep 07 16:26:20.506064 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.506032    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:26:20.569467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.569433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr7bv\" (UniqueName: \"kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv\") pod \"541645af-807f-4310-b0fe-ee5f481c39e7\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") "
Sep 07 16:26:20.569652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.569487    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume\") pod \"541645af-807f-4310-b0fe-ee5f481c39e7\" (UID: \"541645af-807f-4310-b0fe-ee5f481c39e7\") "
Sep 07 16:26:20.569846 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.569823    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "541645af-807f-4310-b0fe-ee5f481c39e7" (UID: "541645af-807f-4310-b0fe-ee5f481c39e7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:26:20.571603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.571577    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv" (OuterVolumeSpecName: "kube-api-access-xr7bv") pod "541645af-807f-4310-b0fe-ee5f481c39e7" (UID: "541645af-807f-4310-b0fe-ee5f481c39e7"). InnerVolumeSpecName "kube-api-access-xr7bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:26:20.670606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.670566    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr7bv\" (UniqueName: \"kubernetes.io/projected/541645af-807f-4310-b0fe-ee5f481c39e7-kube-api-access-xr7bv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:20.670606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:20.670600    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/541645af-807f-4310-b0fe-ee5f481c39e7-extensions-socket-volume\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:21.224968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.224934    2579 generic.go:356] "Generic (PLEG): container finished" podID="541645af-807f-4310-b0fe-ee5f481c39e7" containerID="54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f" exitCode=0
Sep 07 16:26:21.225386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.224981    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"
Sep 07 16:26:21.225386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.225012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" event={"ID":"541645af-807f-4310-b0fe-ee5f481c39e7","Type":"ContainerDied","Data":"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"}
Sep 07 16:26:21.225386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.225064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb" event={"ID":"541645af-807f-4310-b0fe-ee5f481c39e7","Type":"ContainerDied","Data":"deaaaf89d9a15aee3d24c202ef20656b607c73dca495e71424d3b42e88be5f3c"}
Sep 07 16:26:21.225386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.225093    2579 scope.go:117] "RemoveContainer" containerID="54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"
Sep 07 16:26:21.234641 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.234626    2579 scope.go:117] "RemoveContainer" containerID="54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"
Sep 07 16:26:21.234939 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:26:21.234919    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f\": container with ID starting with 54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f not found: ID does not exist" containerID="54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"
Sep 07 16:26:21.234983 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.234946    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f"} err="failed to get container status \"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f\": rpc error: code = NotFound desc = could not find container \"54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f\": container with ID starting with 54ea5ba2f2e638ec769740a405f0281fa57f783c575759b4d3688ab7bc64008f not found: ID does not exist"
Sep 07 16:26:21.247621 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.247597    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:26:21.253256 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.253233    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tcmpb"]
Sep 07 16:26:21.888292 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.888261    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0280167b-4630-4e08-baf8-cc1779249294" path="/var/lib/kubelet/pods/0280167b-4630-4e08-baf8-cc1779249294/volumes"
Sep 07 16:26:21.888609 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:21.888596    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="541645af-807f-4310-b0fe-ee5f481c39e7" path="/var/lib/kubelet/pods/541645af-807f-4310-b0fe-ee5f481c39e7/volumes"
Sep 07 16:26:24.356249 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356218    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"]
Sep 07 16:26:24.356636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356597    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0280167b-4630-4e08-baf8-cc1779249294" containerName="console"
Sep 07 16:26:24.356636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356609    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0280167b-4630-4e08-baf8-cc1779249294" containerName="console"
Sep 07 16:26:24.356636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356630    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="541645af-807f-4310-b0fe-ee5f481c39e7" containerName="manager"
Sep 07 16:26:24.356636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356636    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="541645af-807f-4310-b0fe-ee5f481c39e7" containerName="manager"
Sep 07 16:26:24.356836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356683    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0280167b-4630-4e08-baf8-cc1779249294" containerName="console"
Sep 07 16:26:24.356836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.356695    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="541645af-807f-4310-b0fe-ee5f481c39e7" containerName="manager"
Sep 07 16:26:24.365165 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.365145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.367734 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.367713    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 16:26:24.367896 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.367736    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g5s8p\""
Sep 07 16:26:24.374493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.374469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"]
Sep 07 16:26:24.402759 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53fd93db-cb25-44db-b969-75c23131cd9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.402929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.402929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.402929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.402929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.403088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53fd93db-cb25-44db-b969-75c23131cd9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.403088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.403088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.402988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgb8\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-kube-api-access-bsgb8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.403088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.403041    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53fd93db-cb25-44db-b969-75c23131cd9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.503858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgb8\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-kube-api-access-bsgb8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.503927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53fd93db-cb25-44db-b969-75c23131cd9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.504555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.504537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53fd93db-cb25-44db-b969-75c23131cd9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.506168 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.506147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53fd93db-cb25-44db-b969-75c23131cd9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.506287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.506273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53fd93db-cb25-44db-b969-75c23131cd9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.513062 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.513038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.513281 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.513253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgb8\" (UniqueName: \"kubernetes.io/projected/53fd93db-cb25-44db-b969-75c23131cd9d-kube-api-access-bsgb8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnd5h\" (UID: \"53fd93db-cb25-44db-b969-75c23131cd9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.676215 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.676131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:24.953790 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:24.953739    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"]
Sep 07 16:26:24.955360 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:26:24.955331    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53fd93db_cb25_44db_b969_75c23131cd9d.slice/crio-9799decb1e59a791818f6215523b1966e7452d3ace7490a124f655c3ced89951 WatchSource:0}: Error finding container 9799decb1e59a791818f6215523b1966e7452d3ace7490a124f655c3ced89951: Status 404 returned error can't find the container with id 9799decb1e59a791818f6215523b1966e7452d3ace7490a124f655c3ced89951
Sep 07 16:26:25.248795 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:25.248675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" event={"ID":"53fd93db-cb25-44db-b969-75c23131cd9d","Type":"ContainerStarted","Data":"9799decb1e59a791818f6215523b1966e7452d3ace7490a124f655c3ced89951"}
Sep 07 16:26:27.515236 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:27.515193    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:26:27.515486 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:27.515265    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:26:27.515486 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:27.515294    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:26:28.268640 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:28.268603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" event={"ID":"53fd93db-cb25-44db-b969-75c23131cd9d","Type":"ContainerStarted","Data":"51f4dcd0ce79e7473487373470ce1c152896569d9a51acd1b4bb9d01fdf7456e"}
Sep 07 16:26:28.286339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:28.286269    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podStartSLOduration=1.729169266 podStartE2EDuration="4.286254851s" totalImagesPullingTime="2.557085585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:26:28.2851184 +0000 UTC m=+426.929413731" watchObservedRunningTime="2026-09-07 16:26:28.286254851 +0000 UTC m=+426.930550159"
Sep 07 16:26:28.676604 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:28.676515    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:28.682039 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:28.682017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:29.273800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:29.273734    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:29.274711 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:29.274686    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:26:37.108968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.108930    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:37.112920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.112897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.115459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.115429    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 16:26:37.118200 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.118179    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:37.221520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.215640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:37.222375 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.222344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.222527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.222508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmtns\" (UniqueName: \"kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.323304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.323266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmtns\" (UniqueName: \"kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.323466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.323329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.324028 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.324003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.330604 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.330574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmtns\" (UniqueName: \"kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns\") pod \"limitador-limitador-859c4d96d6-g8sbc\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.423403 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.423308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:37.492222 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.492192    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:26:37.496774 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.496738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.505508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.505482    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:26:37.529249 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.529213    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:26:37.627542 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.627507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-config-file\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.627718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.627568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtr7\" (UniqueName: \"kubernetes.io/projected/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-kube-api-access-5qtr7\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.694064 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.694040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:37.696983 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:26:37.696954    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea399f5b_b11c_4126_a821_7208b8d23dc9.slice/crio-157df6adfa79b76b7fd553c79f149744ef9584e9bafa9d3910656d4e109c988a WatchSource:0}: Error finding container 157df6adfa79b76b7fd553c79f149744ef9584e9bafa9d3910656d4e109c988a: Status 404 returned error can't find the container with id 157df6adfa79b76b7fd553c79f149744ef9584e9bafa9d3910656d4e109c988a
Sep 07 16:26:37.728949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.728911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtr7\" (UniqueName: \"kubernetes.io/projected/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-kube-api-access-5qtr7\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.729119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.729012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-config-file\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.729555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.729535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-config-file\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.739121 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.739089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtr7\" (UniqueName: \"kubernetes.io/projected/3b18d8c6-7df9-42ac-b302-9fff3806eb7a-kube-api-access-5qtr7\") pod \"limitador-limitador-85d9db4d5d-ffvz2\" (UID: \"3b18d8c6-7df9-42ac-b302-9fff3806eb7a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:37.810515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:37.810475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:38.000163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.000124    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:26:38.004772 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.004733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:26:38.007006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.006969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mhh4k\""
Sep 07 16:26:38.010030 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.010001    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:26:38.084021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.083996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:26:38.085478 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:26:38.085450    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b18d8c6_7df9_42ac_b302_9fff3806eb7a.slice/crio-64ab5d0fa10e5621d3b68022dbcbd087c4cdf776a66978a1f24615744192f817 WatchSource:0}: Error finding container 64ab5d0fa10e5621d3b68022dbcbd087c4cdf776a66978a1f24615744192f817: Status 404 returned error can't find the container with id 64ab5d0fa10e5621d3b68022dbcbd087c4cdf776a66978a1f24615744192f817
Sep 07 16:26:38.133603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.133570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvg9h\" (UniqueName: \"kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h\") pod \"authorino-c548b5d7d-qzw4d\" (UID: \"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0\") " pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:26:38.234811 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.234769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvg9h\" (UniqueName: \"kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h\") pod \"authorino-c548b5d7d-qzw4d\" (UID: \"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0\") " pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:26:38.241715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.241692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvg9h\" (UniqueName: \"kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h\") pod \"authorino-c548b5d7d-qzw4d\" (UID: \"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0\") " pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:26:38.316482 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.316396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:26:38.326802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.326769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2" event={"ID":"3b18d8c6-7df9-42ac-b302-9fff3806eb7a","Type":"ContainerStarted","Data":"64ab5d0fa10e5621d3b68022dbcbd087c4cdf776a66978a1f24615744192f817"}
Sep 07 16:26:38.328134 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.328105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" event={"ID":"ea399f5b-b11c-4126-a821-7208b8d23dc9","Type":"ContainerStarted","Data":"157df6adfa79b76b7fd553c79f149744ef9584e9bafa9d3910656d4e109c988a"}
Sep 07 16:26:38.587087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:38.587015    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:26:38.589513 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:26:38.589473    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f142a44_ad1c_42f9_a0f2_83d2f2ae89a0.slice/crio-81d06b05a47bb390c101264a770f98eb373c2e3c4c36423b8d5fadaaaaf08930 WatchSource:0}: Error finding container 81d06b05a47bb390c101264a770f98eb373c2e3c4c36423b8d5fadaaaaf08930: Status 404 returned error can't find the container with id 81d06b05a47bb390c101264a770f98eb373c2e3c4c36423b8d5fadaaaaf08930
Sep 07 16:26:39.335107 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:39.335068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" event={"ID":"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0","Type":"ContainerStarted","Data":"81d06b05a47bb390c101264a770f98eb373c2e3c4c36423b8d5fadaaaaf08930"}
Sep 07 16:26:40.344107 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.344068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2" event={"ID":"3b18d8c6-7df9-42ac-b302-9fff3806eb7a","Type":"ContainerStarted","Data":"644d23ba645089e6f748d1d112978d28ed8b090ff485047fa016bb0fc3f806ba"}
Sep 07 16:26:40.344531 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.344316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:40.345976 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.345951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" event={"ID":"ea399f5b-b11c-4126-a821-7208b8d23dc9","Type":"ContainerStarted","Data":"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"}
Sep 07 16:26:40.346118 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.346077    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:40.358240 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.358192    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2" podStartSLOduration=1.300796213 podStartE2EDuration="3.358177624s" totalImagesPullingTime="2.057381411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:26:40.357119196 +0000 UTC m=+439.001414534" watchObservedRunningTime="2026-09-07 16:26:40.358177624 +0000 UTC m=+439.002473010"
Sep 07 16:26:40.370198 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:40.370147    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" podStartSLOduration=0.926829285 podStartE2EDuration="3.370129019s" totalImagesPullingTime="2.443299734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:26:40.369999701 +0000 UTC m=+439.014295025" watchObservedRunningTime="2026-09-07 16:26:40.370129019 +0000 UTC m=+439.014424329"
Sep 07 16:26:41.353664 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:41.353617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" event={"ID":"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0","Type":"ContainerStarted","Data":"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"}
Sep 07 16:26:51.355235 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:51.355157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:51.355601 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:51.355320    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"
Sep 07 16:26:51.371246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:51.371195    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" podStartSLOduration=12.273782825 podStartE2EDuration="14.371179267s" totalImagesPullingTime="2.097396442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:26:41.365555485 +0000 UTC m=+440.009850795" watchObservedRunningTime="2026-09-07 16:26:51.371179267 +0000 UTC m=+450.015474576"
Sep 07 16:26:51.408913 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:51.408875    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:51.419298 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:51.419264    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" podUID="ea399f5b-b11c-4126-a821-7208b8d23dc9" containerName="limitador" containerID="cri-o://ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18" gracePeriod=30
Sep 07 16:26:52.373045 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.373021    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:52.425719 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.425683    2579 generic.go:356] "Generic (PLEG): container finished" podID="ea399f5b-b11c-4126-a821-7208b8d23dc9" containerID="ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18" exitCode=0
Sep 07 16:26:52.425719 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.425716    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"
Sep 07 16:26:52.425910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.425782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" event={"ID":"ea399f5b-b11c-4126-a821-7208b8d23dc9","Type":"ContainerDied","Data":"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"}
Sep 07 16:26:52.425910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.425857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-g8sbc" event={"ID":"ea399f5b-b11c-4126-a821-7208b8d23dc9","Type":"ContainerDied","Data":"157df6adfa79b76b7fd553c79f149744ef9584e9bafa9d3910656d4e109c988a"}
Sep 07 16:26:52.425910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.425878    2579 scope.go:117] "RemoveContainer" containerID="ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"
Sep 07 16:26:52.434810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.434792    2579 scope.go:117] "RemoveContainer" containerID="ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"
Sep 07 16:26:52.435064 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:26:52.435045    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18\": container with ID starting with ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18 not found: ID does not exist" containerID="ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"
Sep 07 16:26:52.435134 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.435076    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18"} err="failed to get container status \"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18\": rpc error: code = NotFound desc = could not find container \"ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18\": container with ID starting with ea1a762d431ff01e7f35eaa94120efe703d3eda0019da0cc8520037aefdece18 not found: ID does not exist"
Sep 07 16:26:52.464508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.464487    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmtns\" (UniqueName: \"kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns\") pod \"ea399f5b-b11c-4126-a821-7208b8d23dc9\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") "
Sep 07 16:26:52.464588 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.464532    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file\") pod \"ea399f5b-b11c-4126-a821-7208b8d23dc9\" (UID: \"ea399f5b-b11c-4126-a821-7208b8d23dc9\") "
Sep 07 16:26:52.464902 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.464879    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file" (OuterVolumeSpecName: "config-file") pod "ea399f5b-b11c-4126-a821-7208b8d23dc9" (UID: "ea399f5b-b11c-4126-a821-7208b8d23dc9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:26:52.466520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.466496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns" (OuterVolumeSpecName: "kube-api-access-pmtns") pod "ea399f5b-b11c-4126-a821-7208b8d23dc9" (UID: "ea399f5b-b11c-4126-a821-7208b8d23dc9"). InnerVolumeSpecName "kube-api-access-pmtns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:26:52.566021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.565960    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmtns\" (UniqueName: \"kubernetes.io/projected/ea399f5b-b11c-4126-a821-7208b8d23dc9-kube-api-access-pmtns\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:52.566021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.565986    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea399f5b-b11c-4126-a821-7208b8d23dc9-config-file\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:26:52.748454 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.748427    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:52.750586 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:52.750566    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-g8sbc"]
Sep 07 16:26:53.888065 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:26:53.888032    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea399f5b-b11c-4126-a821-7208b8d23dc9" path="/var/lib/kubelet/pods/ea399f5b-b11c-4126-a821-7208b8d23dc9/volumes"
Sep 07 16:27:08.186661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.186624    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xdx6n"]
Sep 07 16:27:08.187088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.187062    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea399f5b-b11c-4126-a821-7208b8d23dc9" containerName="limitador"
Sep 07 16:27:08.187088 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.187074    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea399f5b-b11c-4126-a821-7208b8d23dc9" containerName="limitador"
Sep 07 16:27:08.187166 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.187149    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea399f5b-b11c-4126-a821-7208b8d23dc9" containerName="limitador"
Sep 07 16:27:08.195010 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.194982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.195935 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.195855    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xdx6n"]
Sep 07 16:27:08.275542 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.275507    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xdx6n"]
Sep 07 16:27:08.275821 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:27:08.275801    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pv7bz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n" podUID="f5dc4fa6-001f-4369-abb7-7382f09593c1"
Sep 07 16:27:08.300258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.300223    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-rqw2z"]
Sep 07 16:27:08.304277 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.304251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.304963 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.304944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv7bz\" (UniqueName: \"kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz\") pod \"authorino-5dc7ccb5f-xdx6n\" (UID: \"f5dc4fa6-001f-4369-abb7-7382f09593c1\") " pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.306462 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.306439    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 16:27:08.309635 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.309614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-rqw2z"]
Sep 07 16:27:08.405895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.405866    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-rqw2z"]
Sep 07 16:27:08.406058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.406013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv7bz\" (UniqueName: \"kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz\") pod \"authorino-5dc7ccb5f-xdx6n\" (UID: \"f5dc4fa6-001f-4369-abb7-7382f09593c1\") " pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.406058 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.406048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9dn9\" (UniqueName: \"kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.406153 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:27:08.406093    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-j9dn9 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-rqw2z" podUID="1862b8be-2ee5-4bd0-8f34-36fe9794b71b"
Sep 07 16:27:08.406153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.406100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.416512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.416485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv7bz\" (UniqueName: \"kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz\") pod \"authorino-5dc7ccb5f-xdx6n\" (UID: \"f5dc4fa6-001f-4369-abb7-7382f09593c1\") " pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.507021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.506970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9dn9\" (UniqueName: \"kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.507207 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.507068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.509786 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.509733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.513396 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.513372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.513516 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.513372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.514057 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.514040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9dn9\" (UniqueName: \"kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9\") pod \"authorino-687bd45585-rqw2z\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") " pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.533317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.533297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:08.536819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.536796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:08.708564 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.708532    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert\") pod \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") "
Sep 07 16:27:08.708783 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.708601    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9dn9\" (UniqueName: \"kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9\") pod \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\" (UID: \"1862b8be-2ee5-4bd0-8f34-36fe9794b71b\") "
Sep 07 16:27:08.708783 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.708631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv7bz\" (UniqueName: \"kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz\") pod \"f5dc4fa6-001f-4369-abb7-7382f09593c1\" (UID: \"f5dc4fa6-001f-4369-abb7-7382f09593c1\") "
Sep 07 16:27:08.710986 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.710950    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1862b8be-2ee5-4bd0-8f34-36fe9794b71b" (UID: "1862b8be-2ee5-4bd0-8f34-36fe9794b71b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:27:08.711095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.711005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz" (OuterVolumeSpecName: "kube-api-access-pv7bz") pod "f5dc4fa6-001f-4369-abb7-7382f09593c1" (UID: "f5dc4fa6-001f-4369-abb7-7382f09593c1"). InnerVolumeSpecName "kube-api-access-pv7bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:27:08.711095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.711019    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9" (OuterVolumeSpecName: "kube-api-access-j9dn9") pod "1862b8be-2ee5-4bd0-8f34-36fe9794b71b" (UID: "1862b8be-2ee5-4bd0-8f34-36fe9794b71b"). InnerVolumeSpecName "kube-api-access-j9dn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:27:08.809459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.809365    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-tls-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:08.809459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.809397    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9dn9\" (UniqueName: \"kubernetes.io/projected/1862b8be-2ee5-4bd0-8f34-36fe9794b71b-kube-api-access-j9dn9\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:08.809459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:08.809412    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv7bz\" (UniqueName: \"kubernetes.io/projected/f5dc4fa6-001f-4369-abb7-7382f09593c1-kube-api-access-pv7bz\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:09.518665 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.518625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-rqw2z"
Sep 07 16:27:09.519167 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.518627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xdx6n"
Sep 07 16:27:09.547178 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.547092    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-rqw2z"]
Sep 07 16:27:09.549934 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.549900    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-rqw2z"]
Sep 07 16:27:09.565780 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.565731    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xdx6n"]
Sep 07 16:27:09.569153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.569129    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xdx6n"]
Sep 07 16:27:09.719004 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.718969    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:27:09.719314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.719286    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" podUID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" containerName="authorino" containerID="cri-o://40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6" gracePeriod=30
Sep 07 16:27:09.889487 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.889453    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1862b8be-2ee5-4bd0-8f34-36fe9794b71b" path="/var/lib/kubelet/pods/1862b8be-2ee5-4bd0-8f34-36fe9794b71b/volumes"
Sep 07 16:27:09.889862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.889816    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5dc4fa6-001f-4369-abb7-7382f09593c1" path="/var/lib/kubelet/pods/f5dc4fa6-001f-4369-abb7-7382f09593c1/volumes"
Sep 07 16:27:09.971185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:09.971157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:27:10.120082 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.119991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvg9h\" (UniqueName: \"kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h\") pod \"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0\" (UID: \"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0\") "
Sep 07 16:27:10.122187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.122162    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h" (OuterVolumeSpecName: "kube-api-access-nvg9h") pod "5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" (UID: "5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0"). InnerVolumeSpecName "kube-api-access-nvg9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:27:10.221312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.221277    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvg9h\" (UniqueName: \"kubernetes.io/projected/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0-kube-api-access-nvg9h\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:10.525610 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.525574    2579 generic.go:356] "Generic (PLEG): container finished" podID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" containerID="40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6" exitCode=0
Sep 07 16:27:10.526108 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.525618    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qzw4d"
Sep 07 16:27:10.526108 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.525652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" event={"ID":"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0","Type":"ContainerDied","Data":"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"}
Sep 07 16:27:10.526108 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.525694    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qzw4d" event={"ID":"5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0","Type":"ContainerDied","Data":"81d06b05a47bb390c101264a770f98eb373c2e3c4c36423b8d5fadaaaaf08930"}
Sep 07 16:27:10.526108 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.525710    2579 scope.go:117] "RemoveContainer" containerID="40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"
Sep 07 16:27:10.534574 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.534559    2579 scope.go:117] "RemoveContainer" containerID="40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"
Sep 07 16:27:10.534858 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:27:10.534839    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6\": container with ID starting with 40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6 not found: ID does not exist" containerID="40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"
Sep 07 16:27:10.534913 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.534868    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6"} err="failed to get container status \"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6\": rpc error: code = NotFound desc = could not find container \"40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6\": container with ID starting with 40f30f838a728a2914590031dedd2459470dd7d7750124af835bfe5de25c48b6 not found: ID does not exist"
Sep 07 16:27:10.544483 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.544463    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:27:10.548176 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:10.548157    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qzw4d"]
Sep 07 16:27:11.888596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:11.888559    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" path="/var/lib/kubelet/pods/5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0/volumes"
Sep 07 16:27:14.262994 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.262958    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:14.263361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.263317    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" containerName="authorino"
Sep 07 16:27:14.263361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.263328    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" containerName="authorino"
Sep 07 16:27:14.263436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.263399    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f142a44-ad1c-42f9-a0f2-83d2f2ae89a0" containerName="authorino"
Sep 07 16:27:14.266283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.266266    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.268171 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.268150    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 16:27:14.269072 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.269051    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 16:27:14.269187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.269079    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 16:27:14.269187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.269079    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cvp8t\""
Sep 07 16:27:14.277318 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.277283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:14.350547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.350515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzdw9\" (UniqueName: \"kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.350547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.350549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.350816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.350590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.445424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.445390    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:27:14.448949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.448910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.451326 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwg2\" (UniqueName: \"kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.451441 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.451441 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzdw9\" (UniqueName: \"kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.451441 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.451652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.451652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.451545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.454496 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.454473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.454602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.454473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.456401 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.456380    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:27:14.458943 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.458920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzdw9\" (UniqueName: \"kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9\") pod \"maas-controller-64585c677f-pstcw\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") " pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.552297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.552213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.552297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.552256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwg2\" (UniqueName: \"kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.552297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.552285    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.554784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.554737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.554784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.554780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.559047 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.559026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwg2\" (UniqueName: \"kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2\") pod \"maas-controller-56db7f4c4c-w7nfm\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") " pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.577564 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.577542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:14.776897 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.776860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:14.848711 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:14.848687    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:14.850708 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:27:14.850685    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a0a442_6657_4636_872c_63ca88e75445.slice/crio-a8f930d59280968021c29c67878c91d7f8d5e29a195351ff7d7f2d1ecc4dcbd6 WatchSource:0}: Error finding container a8f930d59280968021c29c67878c91d7f8d5e29a195351ff7d7f2d1ecc4dcbd6: Status 404 returned error can't find the container with id a8f930d59280968021c29c67878c91d7f8d5e29a195351ff7d7f2d1ecc4dcbd6
Sep 07 16:27:15.047788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:15.047765    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:27:15.049865 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:27:15.049836    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a0b68a_29be_4542_96a4_32b4d46ab684.slice/crio-9c062c191ccdda621155e2c4fc04df94f1ad6a1e9e91d2faed7fd969c0bbe683 WatchSource:0}: Error finding container 9c062c191ccdda621155e2c4fc04df94f1ad6a1e9e91d2faed7fd969c0bbe683: Status 404 returned error can't find the container with id 9c062c191ccdda621155e2c4fc04df94f1ad6a1e9e91d2faed7fd969c0bbe683
Sep 07 16:27:15.559717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:15.559634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" event={"ID":"d6a0b68a-29be-4542-96a4-32b4d46ab684","Type":"ContainerStarted","Data":"9c062c191ccdda621155e2c4fc04df94f1ad6a1e9e91d2faed7fd969c0bbe683"}
Sep 07 16:27:15.561980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:15.561944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64585c677f-pstcw" event={"ID":"d3a0a442-6657-4636-872c-63ca88e75445","Type":"ContainerStarted","Data":"a8f930d59280968021c29c67878c91d7f8d5e29a195351ff7d7f2d1ecc4dcbd6"}
Sep 07 16:27:18.582904 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.582855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64585c677f-pstcw" event={"ID":"d3a0a442-6657-4636-872c-63ca88e75445","Type":"ContainerStarted","Data":"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"}
Sep 07 16:27:18.583354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.582925    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:18.584436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.584413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" event={"ID":"d6a0b68a-29be-4542-96a4-32b4d46ab684","Type":"ContainerStarted","Data":"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"}
Sep 07 16:27:18.584527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.584484    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:18.600201 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.600150    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64585c677f-pstcw" podStartSLOduration=1.870329827 podStartE2EDuration="4.600134816s" totalImagesPullingTime="2.729804989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:27:18.598724163 +0000 UTC m=+477.243019494" watchObservedRunningTime="2026-09-07 16:27:18.600134816 +0000 UTC m=+477.244430124"
Sep 07 16:27:18.621727 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:18.621674    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" podStartSLOduration=2.08020581 podStartE2EDuration="4.621660209s" totalImagesPullingTime="2.541454399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:27:18.618989781 +0000 UTC m=+477.263285090" watchObservedRunningTime="2026-09-07 16:27:18.621660209 +0000 UTC m=+477.265955548"
Sep 07 16:27:29.592253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.592215    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:29.592837 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.592310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:27:29.650995 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.650950    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:29.651205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.651176    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64585c677f-pstcw" podUID="d3a0a442-6657-4636-872c-63ca88e75445" containerName="manager" containerID="cri-o://2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032" gracePeriod=30
Sep 07 16:27:29.898783 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.898733    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:29.989400 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.989365    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs\") pod \"d3a0a442-6657-4636-872c-63ca88e75445\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") "
Sep 07 16:27:29.989400 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.989402    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert\") pod \"d3a0a442-6657-4636-872c-63ca88e75445\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") "
Sep 07 16:27:29.989638 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.989439    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzdw9\" (UniqueName: \"kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9\") pod \"d3a0a442-6657-4636-872c-63ca88e75445\" (UID: \"d3a0a442-6657-4636-872c-63ca88e75445\") "
Sep 07 16:27:29.991955 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.991925    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d3a0a442-6657-4636-872c-63ca88e75445" (UID: "d3a0a442-6657-4636-872c-63ca88e75445"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:27:29.991955 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.991939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9" (OuterVolumeSpecName: "kube-api-access-mzdw9") pod "d3a0a442-6657-4636-872c-63ca88e75445" (UID: "d3a0a442-6657-4636-872c-63ca88e75445"). InnerVolumeSpecName "kube-api-access-mzdw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:27:29.991955 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:29.991947    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d3a0a442-6657-4636-872c-63ca88e75445" (UID: "d3a0a442-6657-4636-872c-63ca88e75445"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:27:30.090594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.090549    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:30.090594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.090584    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3a0a442-6657-4636-872c-63ca88e75445-webhook-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:30.090594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.090592    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzdw9\" (UniqueName: \"kubernetes.io/projected/d3a0a442-6657-4636-872c-63ca88e75445-kube-api-access-mzdw9\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:27:30.654112 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.654069    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3a0a442-6657-4636-872c-63ca88e75445" containerID="2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032" exitCode=0
Sep 07 16:27:30.654112 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.654112    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64585c677f-pstcw"
Sep 07 16:27:30.654597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.654129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64585c677f-pstcw" event={"ID":"d3a0a442-6657-4636-872c-63ca88e75445","Type":"ContainerDied","Data":"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"}
Sep 07 16:27:30.654597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.654187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64585c677f-pstcw" event={"ID":"d3a0a442-6657-4636-872c-63ca88e75445","Type":"ContainerDied","Data":"a8f930d59280968021c29c67878c91d7f8d5e29a195351ff7d7f2d1ecc4dcbd6"}
Sep 07 16:27:30.654597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.654211    2579 scope.go:117] "RemoveContainer" containerID="2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"
Sep 07 16:27:30.663834 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.663812    2579 scope.go:117] "RemoveContainer" containerID="2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"
Sep 07 16:27:30.664134 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:27:30.664115    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032\": container with ID starting with 2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032 not found: ID does not exist" containerID="2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"
Sep 07 16:27:30.664186 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.664142    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032"} err="failed to get container status \"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032\": rpc error: code = NotFound desc = could not find container \"2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032\": container with ID starting with 2025fd6e62bc604a05a1e8300d1aca4222ea56e4ee28edfdc8ebd7b545607032 not found: ID does not exist"
Sep 07 16:27:30.673409 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.673268    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:30.675569 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:30.675548    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64585c677f-pstcw"]
Sep 07 16:27:31.889123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:31.889080    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3a0a442-6657-4636-872c-63ca88e75445" path="/var/lib/kubelet/pods/d3a0a442-6657-4636-872c-63ca88e75445/volumes"
Sep 07 16:27:51.853424 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.853385    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:27:51.853844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.853829    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a0a442-6657-4636-872c-63ca88e75445" containerName="manager"
Sep 07 16:27:51.853897 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.853846    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a0a442-6657-4636-872c-63ca88e75445" containerName="manager"
Sep 07 16:27:51.853933 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.853908    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a0a442-6657-4636-872c-63ca88e75445" containerName="manager"
Sep 07 16:27:51.860426 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.860405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:51.863705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.863679    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wj7bk\""
Sep 07 16:27:51.863913 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.863882    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 16:27:51.864036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.863817    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 16:27:51.864147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.863828    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 16:27:51.864147 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.863733    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 16:27:51.867097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.867075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:27:51.987475 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.987448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:51.987475 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.987482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:51.987733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.987519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4h96\" (UniqueName: \"kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:51.987733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:51.987631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.088561 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.088525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4h96\" (UniqueName: \"kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.088729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.088571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.088729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.088624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.088729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.088667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.089197 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.089169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.091200 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.091177    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.091200 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.091190    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.097114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.097091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4h96\" (UniqueName: \"kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96\") pod \"maas-api-5868b9dbc4-vlkhc\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") " pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.172431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.172354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:52.425736 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.425641    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"]
Sep 07 16:27:52.430738 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.430714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.432947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.432918    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 07 16:27:52.433066 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.432931    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nkfbn\""
Sep 07 16:27:52.441519 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.441493    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"]
Sep 07 16:27:52.457678 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.457652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:27:52.459399 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:27:52.459369    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6726ca22_7c51_4be7_81e3_23d3b6821a55.slice/crio-d3d7e88678e1514e36df0da4c6d432e75255bca1cb99fd9c0fa32c3b58e89b44 WatchSource:0}: Error finding container d3d7e88678e1514e36df0da4c6d432e75255bca1cb99fd9c0fa32c3b58e89b44: Status 404 returned error can't find the container with id d3d7e88678e1514e36df0da4c6d432e75255bca1cb99fd9c0fa32c3b58e89b44
Sep 07 16:27:52.466203 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.466182    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:27:52.594217 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.594180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mrj\" (UniqueName: \"kubernetes.io/projected/b86846dd-38ae-484c-acb6-5753fa726611-kube-api-access-92mrj\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.594414 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.594248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86846dd-38ae-484c-acb6-5753fa726611-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.695959 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.695868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mrj\" (UniqueName: \"kubernetes.io/projected/b86846dd-38ae-484c-acb6-5753fa726611-kube-api-access-92mrj\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.696139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.695958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86846dd-38ae-484c-acb6-5753fa726611-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.696702 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.696678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86846dd-38ae-484c-acb6-5753fa726611-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.703783 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.703732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mrj\" (UniqueName: \"kubernetes.io/projected/b86846dd-38ae-484c-acb6-5753fa726611-kube-api-access-92mrj\") pod \"payload-processing-5b54b7fbb7-sz5qv\" (UID: \"b86846dd-38ae-484c-acb6-5753fa726611\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.743559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.743523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:52.785376 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:52.785339    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" event={"ID":"6726ca22-7c51-4be7-81e3-23d3b6821a55","Type":"ContainerStarted","Data":"d3d7e88678e1514e36df0da4c6d432e75255bca1cb99fd9c0fa32c3b58e89b44"}
Sep 07 16:27:53.039131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:53.039104    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"]
Sep 07 16:27:53.040161 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:27:53.040138    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86846dd_38ae_484c_acb6_5753fa726611.slice/crio-acce50cca502239fb372afaee6444a1015e79a2fd6feb31e0d98472d568c8f16 WatchSource:0}: Error finding container acce50cca502239fb372afaee6444a1015e79a2fd6feb31e0d98472d568c8f16: Status 404 returned error can't find the container with id acce50cca502239fb372afaee6444a1015e79a2fd6feb31e0d98472d568c8f16
Sep 07 16:27:53.811158 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:53.811067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv" event={"ID":"b86846dd-38ae-484c-acb6-5753fa726611","Type":"ContainerStarted","Data":"acce50cca502239fb372afaee6444a1015e79a2fd6feb31e0d98472d568c8f16"}
Sep 07 16:27:55.826395 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.826360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv" event={"ID":"b86846dd-38ae-484c-acb6-5753fa726611","Type":"ContainerStarted","Data":"ab24dc300e2e9c3b7600e8322eb4407f3ab0239507be24f5ebb3e63f7d902ba5"}
Sep 07 16:27:55.827045 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.827028    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:27:55.828292 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.828272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" event={"ID":"6726ca22-7c51-4be7-81e3-23d3b6821a55","Type":"ContainerStarted","Data":"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"}
Sep 07 16:27:55.828658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.828644    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:27:55.842361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.842309    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv" podStartSLOduration=1.732350534 podStartE2EDuration="3.842294894s" totalImagesPullingTime="2.10994436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:27:55.840936407 +0000 UTC m=+514.485231716" watchObservedRunningTime="2026-09-07 16:27:55.842294894 +0000 UTC m=+514.486590202"
Sep 07 16:27:55.858519 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:27:55.858463    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" podStartSLOduration=2.173643225 podStartE2EDuration="4.858445202s" totalImagesPullingTime="2.684801977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:27:55.855825936 +0000 UTC m=+514.500121271" watchObservedRunningTime="2026-09-07 16:27:55.858445202 +0000 UTC m=+514.502740510"
Sep 07 16:28:00.314954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:00.314914    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnd5h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 16:28:00.315334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:00.315015    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podUID="53fd93db-cb25-44db-b969-75c23131cd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 16:28:01.867160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.867085    2579 generic.go:356] "Generic (PLEG): container finished" podID="53fd93db-cb25-44db-b969-75c23131cd9d" containerID="51f4dcd0ce79e7473487373470ce1c152896569d9a51acd1b4bb9d01fdf7456e" exitCode=0
Sep 07 16:28:01.867510 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.867158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" event={"ID":"53fd93db-cb25-44db-b969-75c23131cd9d","Type":"ContainerDied","Data":"51f4dcd0ce79e7473487373470ce1c152896569d9a51acd1b4bb9d01fdf7456e"}
Sep 07 16:28:01.867570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.867555    2579 scope.go:117] "RemoveContainer" containerID="51f4dcd0ce79e7473487373470ce1c152896569d9a51acd1b4bb9d01fdf7456e"
Sep 07 16:28:01.868642 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.868615    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:28:01.868708 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.868680    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:28:01.868764 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:01.868712    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 16:28:02.846796 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:02.846769    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:28:02.888149 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:02.888109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" event={"ID":"53fd93db-cb25-44db-b969-75c23131cd9d","Type":"ContainerStarted","Data":"7e0c0c657672a2c9a152c125e1ee52f13d47d8a9274e762d7711504030ef717e"}
Sep 07 16:28:03.677247 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:03.677212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:28:03.677646 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:03.677556    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnd5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 07 16:28:03.677852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:03.677824    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podUID="53fd93db-cb25-44db-b969-75c23131cd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 07 16:28:04.676458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:04.676418    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnd5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 07 16:28:04.676884 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:04.676497    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podUID="53fd93db-cb25-44db-b969-75c23131cd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 07 16:28:05.676669 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:05.676630    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnd5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 07 16:28:05.677154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:05.676718    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podUID="53fd93db-cb25-44db-b969-75c23131cd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 07 16:28:06.676997 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:06.676956    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnd5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 07 16:28:06.677595 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:06.677560    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h" podUID="53fd93db-cb25-44db-b969-75c23131cd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 07 16:28:07.679966 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:07.679933    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:28:07.680370 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:07.680322    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:28:07.681031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:07.681010    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnd5h"
Sep 07 16:28:07.842411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:07.842382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-sz5qv"
Sep 07 16:28:14.040877 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.040786    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"]
Sep 07 16:28:14.044479 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.044459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.047259 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.047237    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-stl6r\""
Sep 07 16:28:14.047390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.047363    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 16:28:14.047465 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.047237    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 07 16:28:14.047555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.047294    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 16:28:14.052290 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.052229    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"]
Sep 07 16:28:14.115002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.114958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.115195 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.115013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqx45\" (UniqueName: \"kubernetes.io/projected/4f5751b3-8131-47bb-bf90-6534a76141b6-kube-api-access-cqx45\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.115195 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.115104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.115195 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.115144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5751b3-8131-47bb-bf90-6534a76141b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.115333 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.115207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.115333 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.115241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216369 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216369 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216617 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216617 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqx45\" (UniqueName: \"kubernetes.io/projected/4f5751b3-8131-47bb-bf90-6534a76141b6-kube-api-access-cqx45\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216617 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216799 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5751b3-8131-47bb-bf90-6534a76141b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216893 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.216945 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.217007 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.216959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.218968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.218937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5751b3-8131-47bb-bf90-6534a76141b6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.219200 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.219180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5751b3-8131-47bb-bf90-6534a76141b6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.223583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.223561    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqx45\" (UniqueName: \"kubernetes.io/projected/4f5751b3-8131-47bb-bf90-6534a76141b6-kube-api-access-cqx45\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh\" (UID: \"4f5751b3-8131-47bb-bf90-6534a76141b6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.356393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.356309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:14.528173 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.528130    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:28:14.634059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.634033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"]
Sep 07 16:28:14.636026 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:28:14.635984    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5751b3_8131_47bb_bf90_6534a76141b6.slice/crio-c062e07c3eaba541376501b4b3595d074f480c16f97551f58a42380d1fc95a55 WatchSource:0}: Error finding container c062e07c3eaba541376501b4b3595d074f480c16f97551f58a42380d1fc95a55: Status 404 returned error can't find the container with id c062e07c3eaba541376501b4b3595d074f480c16f97551f58a42380d1fc95a55
Sep 07 16:28:14.970028 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:14.969984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh" event={"ID":"4f5751b3-8131-47bb-bf90-6534a76141b6","Type":"ContainerStarted","Data":"c062e07c3eaba541376501b4b3595d074f480c16f97551f58a42380d1fc95a55"}
Sep 07 16:28:15.977700 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:15.977655    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh" event={"ID":"4f5751b3-8131-47bb-bf90-6534a76141b6","Type":"ContainerStarted","Data":"33b819aafe80853f55f9a63f820fe73143b081df268ea4ae8919d77788966dce"}
Sep 07 16:28:15.995658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:15.995597    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh" podStartSLOduration=0.788477854 podStartE2EDuration="1.995573294s" totalImagesPullingTime="1.20709544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:28:15.99525546 +0000 UTC m=+534.639550811" watchObservedRunningTime="2026-09-07 16:28:15.995573294 +0000 UTC m=+534.639868603"
Sep 07 16:28:23.128090 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:23.128054    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:28:24.357543 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:24.357502    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:24.358040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:24.357582    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:24.370113 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:24.370082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:25.039350 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:25.039320    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh"
Sep 07 16:28:50.047641 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:28:50.047608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:29:05.771814 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:05.771774    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:29:16.757902 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:16.757868    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:29:21.832164 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:21.832135    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:29:21.832715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:21.832695    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:29:26.533211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:26.533177    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:29:37.437879 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.437844    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"]
Sep 07 16:29:37.441582 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.441561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.443927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.443900    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 07 16:29:37.449652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.449627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"]
Sep 07 16:29:37.472143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b911400-3f32-4d79-a82d-5b14662a16b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.472305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gts8j\" (UniqueName: \"kubernetes.io/projected/4b911400-3f32-4d79-a82d-5b14662a16b2-kube-api-access-gts8j\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.472305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472178    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.472305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.472471 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.472471 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.472381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573039 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.572995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573234 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b911400-3f32-4d79-a82d-5b14662a16b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573234 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gts8j\" (UniqueName: \"kubernetes.io/projected/4b911400-3f32-4d79-a82d-5b14662a16b2-kube-api-access-gts8j\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573234 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573234 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573558 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.573615 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.573598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.575526 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.575508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b911400-3f32-4d79-a82d-5b14662a16b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.575810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.575796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b911400-3f32-4d79-a82d-5b14662a16b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.581661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.581623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gts8j\" (UniqueName: \"kubernetes.io/projected/4b911400-3f32-4d79-a82d-5b14662a16b2-kube-api-access-gts8j\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h\" (UID: \"4b911400-3f32-4d79-a82d-5b14662a16b2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:37.753694 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:37.753638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:38.032224 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:38.032194    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"]
Sep 07 16:29:38.034031 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:29:38.033997    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b911400_3f32_4d79_a82d_5b14662a16b2.slice/crio-4995db33dcfa64fdbb8aafe95ac78b8140f7a9d06117c56144afde01118e9cde WatchSource:0}: Error finding container 4995db33dcfa64fdbb8aafe95ac78b8140f7a9d06117c56144afde01118e9cde: Status 404 returned error can't find the container with id 4995db33dcfa64fdbb8aafe95ac78b8140f7a9d06117c56144afde01118e9cde
Sep 07 16:29:38.035303 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:38.035286    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:29:38.457836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:38.457802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h" event={"ID":"4b911400-3f32-4d79-a82d-5b14662a16b2","Type":"ContainerStarted","Data":"aab93d274f3287c740872929fe6e77e5e2cb7ec11155fe12dcee7ee8411e4466"}
Sep 07 16:29:38.458220 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:38.457850    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h" event={"ID":"4b911400-3f32-4d79-a82d-5b14662a16b2","Type":"ContainerStarted","Data":"4995db33dcfa64fdbb8aafe95ac78b8140f7a9d06117c56144afde01118e9cde"}
Sep 07 16:29:38.474418 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:38.474371    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h" podStartSLOduration=1.294982186 podStartE2EDuration="1.474354552s" totalImagesPullingTime="179.372366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:29:38.47276987 +0000 UTC m=+617.117065173" watchObservedRunningTime="2026-09-07 16:29:38.474354552 +0000 UTC m=+617.118649861"
Sep 07 16:29:39.833390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:39.833336    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:29:47.754474 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:47.754386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:47.754474 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:47.754447    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:47.766809 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:47.766784    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:29:48.528713 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:29:48.528683    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h"
Sep 07 16:30:44.198270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.198235    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:44.201802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.201779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:44.203971 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.203945    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9f7r\""
Sep 07 16:30:44.207460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.207433    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:44.278231 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.278186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5lhv\" (UniqueName: \"kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv\") pod \"test-curl-7796-140428895200432\" (UID: \"894addf2-540b-4aef-9ed0-35747c02e2c3\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:44.379544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.379490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5lhv\" (UniqueName: \"kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv\") pod \"test-curl-7796-140428895200432\" (UID: \"894addf2-540b-4aef-9ed0-35747c02e2c3\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:44.387533 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.387507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5lhv\" (UniqueName: \"kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv\") pod \"test-curl-7796-140428895200432\" (UID: \"894addf2-540b-4aef-9ed0-35747c02e2c3\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:44.512853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.512772    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:44.788042 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.788010    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:44.790077 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:30:44.790045    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod894addf2_540b_4aef_9ed0_35747c02e2c3.slice/crio-2f883e9a3c06fd1441380390b857c16392df247768e1c14731ddc55a6fc706fd WatchSource:0}: Error finding container 2f883e9a3c06fd1441380390b857c16392df247768e1c14731ddc55a6fc706fd: Status 404 returned error can't find the container with id 2f883e9a3c06fd1441380390b857c16392df247768e1c14731ddc55a6fc706fd
Sep 07 16:30:44.853155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:44.853118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-140428895200432" event={"ID":"894addf2-540b-4aef-9ed0-35747c02e2c3","Type":"ContainerStarted","Data":"2f883e9a3c06fd1441380390b857c16392df247768e1c14731ddc55a6fc706fd"}
Sep 07 16:30:45.860007 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:45.859922    2579 generic.go:356] "Generic (PLEG): container finished" podID="894addf2-540b-4aef-9ed0-35747c02e2c3" containerID="e522bcad00d8a6d998d53796567069d26e8f8be06be9e949288922563ee05aba" exitCode=0
Sep 07 16:30:45.860356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:45.860008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-140428895200432" event={"ID":"894addf2-540b-4aef-9ed0-35747c02e2c3","Type":"ContainerDied","Data":"e522bcad00d8a6d998d53796567069d26e8f8be06be9e949288922563ee05aba"}
Sep 07 16:30:46.131428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:46.131341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:30:47.023918 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.023889    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.053667 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.053638    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7796-140428895200432_894addf2-540b-4aef-9ed0-35747c02e2c3/test-curl-7796-140428895200432/0.log"
Sep 07 16:30:47.072835 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.072800    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:47.075104 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.075077    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:47.110693 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.110651    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5lhv\" (UniqueName: \"kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv\") pod \"894addf2-540b-4aef-9ed0-35747c02e2c3\" (UID: \"894addf2-540b-4aef-9ed0-35747c02e2c3\") "
Sep 07 16:30:47.112831 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.112805    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv" (OuterVolumeSpecName: "kube-api-access-g5lhv") pod "894addf2-540b-4aef-9ed0-35747c02e2c3" (UID: "894addf2-540b-4aef-9ed0-35747c02e2c3"). InnerVolumeSpecName "kube-api-access-g5lhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:30:47.186519 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.186477    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:47.186928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.186908    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894addf2-540b-4aef-9ed0-35747c02e2c3" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:47.186928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.186926    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="894addf2-540b-4aef-9ed0-35747c02e2c3" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:47.187101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.187032    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="894addf2-540b-4aef-9ed0-35747c02e2c3" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:47.190205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.190183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.195059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.195033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:47.211732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.211697    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5lhv\" (UniqueName: \"kubernetes.io/projected/894addf2-540b-4aef-9ed0-35747c02e2c3-kube-api-access-g5lhv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:30:47.312404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.312374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scflp\" (UniqueName: \"kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp\") pod \"test-curl-7796-140428895200432\" (UID: \"6879da0b-2c3c-45fe-82b0-5a2a083606ba\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.413809 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.413695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scflp\" (UniqueName: \"kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp\") pod \"test-curl-7796-140428895200432\" (UID: \"6879da0b-2c3c-45fe-82b0-5a2a083606ba\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.421775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.421712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scflp\" (UniqueName: \"kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp\") pod \"test-curl-7796-140428895200432\" (UID: \"6879da0b-2c3c-45fe-82b0-5a2a083606ba\") " pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.502288 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.502253    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.782089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.782060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:47.783953 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:30:47.783928    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6879da0b_2c3c_45fe_82b0_5a2a083606ba.slice/crio-12b417f457cf7488feb8850530dfb92d2c367eab603a69e6420e7c97c75e5070 WatchSource:0}: Error finding container 12b417f457cf7488feb8850530dfb92d2c367eab603a69e6420e7c97c75e5070: Status 404 returned error can't find the container with id 12b417f457cf7488feb8850530dfb92d2c367eab603a69e6420e7c97c75e5070
Sep 07 16:30:47.874091 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.874063    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f883e9a3c06fd1441380390b857c16392df247768e1c14731ddc55a6fc706fd"
Sep 07 16:30:47.874091 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.874067    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:47.875349 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.875319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-140428895200432" event={"ID":"6879da0b-2c3c-45fe-82b0-5a2a083606ba","Type":"ContainerStarted","Data":"12b417f457cf7488feb8850530dfb92d2c367eab603a69e6420e7c97c75e5070"}
Sep 07 16:30:47.891656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.891612    2579 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7796-140428895200432" oldPodUID="894addf2-540b-4aef-9ed0-35747c02e2c3" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba"
Sep 07 16:30:47.894718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:47.894687    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894addf2-540b-4aef-9ed0-35747c02e2c3" path="/var/lib/kubelet/pods/894addf2-540b-4aef-9ed0-35747c02e2c3/volumes"
Sep 07 16:30:48.884501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:48.884471    2579 generic.go:356] "Generic (PLEG): container finished" podID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" containerID="07488c3ac816e60dc0413b390bc81674f5f73e32528cb0ae40aba002ec3a8329" exitCode=0
Sep 07 16:30:48.884898 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:48.884560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-140428895200432" event={"ID":"6879da0b-2c3c-45fe-82b0-5a2a083606ba","Type":"ContainerDied","Data":"07488c3ac816e60dc0413b390bc81674f5f73e32528cb0ae40aba002ec3a8329"}
Sep 07 16:30:49.830077 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.830034    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:30:49.834177 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.834149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:49.836370 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.836348    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-747d6d-serving-cert\""
Sep 07 16:30:49.846433 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.845775    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:30:49.940995 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.940953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:49.942123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.942083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66jx4\" (UniqueName: \"kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:49.942264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.942213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:49.942264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:49.942258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.043308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.043283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.043478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.043318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.043478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.043376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.043478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.043392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66jx4\" (UniqueName: \"kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.044127 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.044098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.046103 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.046078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.046200 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.046140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.048284 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.048268    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:50.051448 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.051427    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66jx4\" (UniqueName: \"kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4\") pod \"maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.065354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.065312    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7796-140428895200432_6879da0b-2c3c-45fe-82b0-5a2a083606ba/test-curl-7796-140428895200432/0.log"
Sep 07 16:30:50.093551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.093458    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:50.095921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.095900    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7796-140428895200432"]
Sep 07 16:30:50.143970 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.143934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scflp\" (UniqueName: \"kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp\") pod \"6879da0b-2c3c-45fe-82b0-5a2a083606ba\" (UID: \"6879da0b-2c3c-45fe-82b0-5a2a083606ba\") "
Sep 07 16:30:50.146022 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.145994    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp" (OuterVolumeSpecName: "kube-api-access-scflp") pod "6879da0b-2c3c-45fe-82b0-5a2a083606ba" (UID: "6879da0b-2c3c-45fe-82b0-5a2a083606ba"). InnerVolumeSpecName "kube-api-access-scflp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:30:50.148216 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.148193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.245271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.245237    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scflp\" (UniqueName: \"kubernetes.io/projected/6879da0b-2c3c-45fe-82b0-5a2a083606ba-kube-api-access-scflp\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:30:50.433991 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.433962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:30:50.436729 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:30:50.436697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30abe198_977f_47ab_97a2_0f1019772840.slice/crio-9e17aebb3b2c0de62d6ec97d18c03bb2987bdad40783f419c607f8d8c851d3a0 WatchSource:0}: Error finding container 9e17aebb3b2c0de62d6ec97d18c03bb2987bdad40783f419c607f8d8c851d3a0: Status 404 returned error can't find the container with id 9e17aebb3b2c0de62d6ec97d18c03bb2987bdad40783f419c607f8d8c851d3a0
Sep 07 16:30:50.899188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.899146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" event={"ID":"30abe198-977f-47ab-97a2-0f1019772840","Type":"ContainerStarted","Data":"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"}
Sep 07 16:30:50.899380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.899220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" event={"ID":"30abe198-977f-47ab-97a2-0f1019772840","Type":"ContainerStarted","Data":"9e17aebb3b2c0de62d6ec97d18c03bb2987bdad40783f419c607f8d8c851d3a0"}
Sep 07 16:30:50.899380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.899368    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:50.900594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.900572    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:50.900710 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.900613    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12b417f457cf7488feb8850530dfb92d2c367eab603a69e6420e7c97c75e5070"
Sep 07 16:30:50.918494 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:50.918445    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" podStartSLOduration=1.569608182 podStartE2EDuration="1.918429935s" totalImagesPullingTime="348.821753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:30:50.917112478 +0000 UTC m=+689.561407810" watchObservedRunningTime="2026-09-07 16:30:50.918429935 +0000 UTC m=+689.562725277"
Sep 07 16:30:50.919149 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:30:50.919119    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7796-140428895200432\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:51.886572 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:30:51.886531    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7796-140428895200432\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" pod="opendatahub/test-curl-7796-140428895200432"
Sep 07 16:30:51.897477 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:51.897440    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" path="/var/lib/kubelet/pods/6879da0b-2c3c-45fe-82b0-5a2a083606ba/volumes"
Sep 07 16:30:55.267648 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.267615    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:30:55.268099 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.268081    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:55.268151 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.268102    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:55.268205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.268195    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6879da0b-2c3c-45fe-82b0-5a2a083606ba" containerName="test-curl-7796-140428895200432"
Sep 07 16:30:55.282596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.282562    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:30:55.282811 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.282646    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.285078 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.285055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-8a3403-serving-cert\""
Sep 07 16:30:55.393542 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.393511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.393788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.393571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.393788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.393596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r845c\" (UniqueName: \"kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.393788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.393686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.494515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.494478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.494515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.494515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r845c\" (UniqueName: \"kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.494844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.494559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.494844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.494620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.495363 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.495308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.497213 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.497183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.497305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.497242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.505399 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.505375    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r845c\" (UniqueName: \"kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c\") pod \"maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.597239 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.597145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:55.874428 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.874389    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:30:55.874911 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:30:55.874888    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de2bdd5_9482_4647_88ac_bbf924189796.slice/crio-930a266cd4820d3aabc262291aa424ed34c53d1af8edee6213f13f6e324a6d95 WatchSource:0}: Error finding container 930a266cd4820d3aabc262291aa424ed34c53d1af8edee6213f13f6e324a6d95: Status 404 returned error can't find the container with id 930a266cd4820d3aabc262291aa424ed34c53d1af8edee6213f13f6e324a6d95
Sep 07 16:30:55.935686 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:55.935642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" event={"ID":"2de2bdd5-9482-4647-88ac-bbf924189796","Type":"ContainerStarted","Data":"930a266cd4820d3aabc262291aa424ed34c53d1af8edee6213f13f6e324a6d95"}
Sep 07 16:30:56.137163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.137081    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:30:56.142269 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.142244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.144627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.144602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-983d9f00-serving-cert\""
Sep 07 16:30:56.151886 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.151843    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:30:56.201063 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.201026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.201063 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.201068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.201267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.201125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf29t\" (UniqueName: \"kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.201267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.201175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.302157 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.302127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.302610 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.302163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.302610 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.302188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf29t\" (UniqueName: \"kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.302610 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.302214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.302802 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.302781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.304819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.304797    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.304819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.304813    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.309205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.309182    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf29t\" (UniqueName: \"kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t\") pod \"maas-api-e2e-ait-983d9f00-6d4554b87b-szz66\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.456628 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.456590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:56.742857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.742829    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:30:56.744826 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:30:56.744796    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46663950_a531_436a_8161_67d2985155a4.slice/crio-bdbc583530c622b4de27db349d3760ad6ad9eff7635ff1bede7b18a056512da5 WatchSource:0}: Error finding container bdbc583530c622b4de27db349d3760ad6ad9eff7635ff1bede7b18a056512da5: Status 404 returned error can't find the container with id bdbc583530c622b4de27db349d3760ad6ad9eff7635ff1bede7b18a056512da5
Sep 07 16:30:56.911523 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.911490    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:30:56.945126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.945088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" event={"ID":"2de2bdd5-9482-4647-88ac-bbf924189796","Type":"ContainerStarted","Data":"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"}
Sep 07 16:30:56.946912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.946884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" event={"ID":"46663950-a531-436a-8161-67d2985155a4","Type":"ContainerStarted","Data":"bdbc583530c622b4de27db349d3760ad6ad9eff7635ff1bede7b18a056512da5"}
Sep 07 16:30:56.967582 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:56.967534    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" podStartSLOduration=1.486386341 podStartE2EDuration="1.967518205s" totalImagesPullingTime="481.131864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:30:56.966856299 +0000 UTC m=+695.611151629" watchObservedRunningTime="2026-09-07 16:30:56.967518205 +0000 UTC m=+695.611813513"
Sep 07 16:30:57.954585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:57.954538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" event={"ID":"46663950-a531-436a-8161-67d2985155a4","Type":"ContainerStarted","Data":"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"}
Sep 07 16:30:57.955008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:57.954603    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:30:57.955008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:57.954921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:30:57.971942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:57.971895    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" podStartSLOduration=1.677210871 podStartE2EDuration="1.971881472s" totalImagesPullingTime="294.670601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:30:57.969567772 +0000 UTC m=+696.613863080" watchObservedRunningTime="2026-09-07 16:30:57.971881472 +0000 UTC m=+696.616176780"
Sep 07 16:30:58.551534 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:30:58.551496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:31:03.965952 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:03.965922    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:31:03.966442 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:03.966342    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:31:12.724882 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:12.724787    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:31:18.928175 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:18.928129    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:31:18.932059 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:18.932037    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:18.933994 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:18.933973    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a7d399-serving-cert\""
Sep 07 16:31:18.943139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:18.943070    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:31:19.126478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.126437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.126478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.126474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.126698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.126496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdtf9\" (UniqueName: \"kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.126698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.126669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.227721 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.227684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.227721 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.227727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.228009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.227769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.228009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.227792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdtf9\" (UniqueName: \"kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.228009 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:31:19.227864    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-serving-cert: secret "maas-api-e2e-inf-b-a7d399-serving-cert" not found
Sep 07 16:31:19.228009 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:31:19.227956    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls podName:c193c0fb-99a0-458c-a41d-0354bea3060a nodeName:}" failed. No retries permitted until 2026-09-07 16:31:19.727934599 +0000 UTC m=+718.372229886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls") pod "maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" (UID: "c193c0fb-99a0-458c-a41d-0354bea3060a") : secret "maas-api-e2e-inf-b-a7d399-serving-cert" not found
Sep 07 16:31:19.228352 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.228334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.230399 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.230379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.239494 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.239466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdtf9\" (UniqueName: \"kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.732852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.732805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.735446 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.735412    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:19.845830 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:19.845787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:20.125456 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:20.125421    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:31:20.128173 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:31:20.128136    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc193c0fb_99a0_458c_a41d_0354bea3060a.slice/crio-77c3e349e55366c0eacb7000e0efef09a65800546710b88bc8fb3d3107b07cca WatchSource:0}: Error finding container 77c3e349e55366c0eacb7000e0efef09a65800546710b88bc8fb3d3107b07cca: Status 404 returned error can't find the container with id 77c3e349e55366c0eacb7000e0efef09a65800546710b88bc8fb3d3107b07cca
Sep 07 16:31:21.097257 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:21.097219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" event={"ID":"c193c0fb-99a0-458c-a41d-0354bea3060a","Type":"ContainerStarted","Data":"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"}
Sep 07 16:31:21.097460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:21.097271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" event={"ID":"c193c0fb-99a0-458c-a41d-0354bea3060a","Type":"ContainerStarted","Data":"77c3e349e55366c0eacb7000e0efef09a65800546710b88bc8fb3d3107b07cca"}
Sep 07 16:31:21.097460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:21.097341    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:21.114321 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:21.114277    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" podStartSLOduration=2.790249475 podStartE2EDuration="3.114262805s" totalImagesPullingTime="324.01333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:31:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:31:21.112383776 +0000 UTC m=+719.756679103" watchObservedRunningTime="2026-09-07 16:31:21.114262805 +0000 UTC m=+719.758558175"
Sep 07 16:31:27.107980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:27.107950    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:31:28.041462 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:28.041419    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:31:36.155014 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.154975    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:31:36.155553 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.155241    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" podUID="46663950-a531-436a-8161-67d2985155a4" containerName="maas-api" containerID="cri-o://1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd" gracePeriod=30
Sep 07 16:31:36.423738 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.423703    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:31:36.492051 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.492013    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs\") pod \"46663950-a531-436a-8161-67d2985155a4\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") "
Sep 07 16:31:36.492224 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.492106    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls\") pod \"46663950-a531-436a-8161-67d2985155a4\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") "
Sep 07 16:31:36.492224 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.492171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf29t\" (UniqueName: \"kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t\") pod \"46663950-a531-436a-8161-67d2985155a4\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") "
Sep 07 16:31:36.492359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.492240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca\") pod \"46663950-a531-436a-8161-67d2985155a4\" (UID: \"46663950-a531-436a-8161-67d2985155a4\") "
Sep 07 16:31:36.492910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.492856    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46663950-a531-436a-8161-67d2985155a4" (UID: "46663950-a531-436a-8161-67d2985155a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:31:36.495047 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.495017    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46663950-a531-436a-8161-67d2985155a4" (UID: "46663950-a531-436a-8161-67d2985155a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:31:36.495431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.495389    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46663950-a531-436a-8161-67d2985155a4" (UID: "46663950-a531-436a-8161-67d2985155a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:31:36.495562 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.495534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t" (OuterVolumeSpecName: "kube-api-access-gf29t") pod "46663950-a531-436a-8161-67d2985155a4" (UID: "46663950-a531-436a-8161-67d2985155a4"). InnerVolumeSpecName "kube-api-access-gf29t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:31:36.593947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.593909    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46663950-a531-436a-8161-67d2985155a4-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:31:36.593947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.593940    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:31:36.593947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.593949    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46663950-a531-436a-8161-67d2985155a4-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:31:36.593947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:36.593957    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf29t\" (UniqueName: \"kubernetes.io/projected/46663950-a531-436a-8161-67d2985155a4-kube-api-access-gf29t\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:31:37.199387 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.199346    2579 generic.go:356] "Generic (PLEG): container finished" podID="46663950-a531-436a-8161-67d2985155a4" containerID="1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd" exitCode=0
Sep 07 16:31:37.199848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.199395    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"
Sep 07 16:31:37.199848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.199435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" event={"ID":"46663950-a531-436a-8161-67d2985155a4","Type":"ContainerDied","Data":"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"}
Sep 07 16:31:37.199848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.199492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66" event={"ID":"46663950-a531-436a-8161-67d2985155a4","Type":"ContainerDied","Data":"bdbc583530c622b4de27db349d3760ad6ad9eff7635ff1bede7b18a056512da5"}
Sep 07 16:31:37.199848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.199517    2579 scope.go:117] "RemoveContainer" containerID="1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"
Sep 07 16:31:37.210515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.210492    2579 scope.go:117] "RemoveContainer" containerID="1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"
Sep 07 16:31:37.210864 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:31:37.210845    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd\": container with ID starting with 1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd not found: ID does not exist" containerID="1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"
Sep 07 16:31:37.210921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.210874    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd"} err="failed to get container status \"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd\": rpc error: code = NotFound desc = could not find container \"1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd\": container with ID starting with 1a6c6288bfcaa97322a05ef4f57c7c2296fa0f0dfaf6da6f78e07de01ed77afd not found: ID does not exist"
Sep 07 16:31:37.240174 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.240132    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:31:37.251735 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.251699    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-983d9f00-6d4554b87b-szz66"]
Sep 07 16:31:37.890180 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:37.890145    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46663950-a531-436a-8161-67d2985155a4" path="/var/lib/kubelet/pods/46663950-a531-436a-8161-67d2985155a4/volumes"
Sep 07 16:31:41.054433 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.054396    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:31:41.055105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.055083    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46663950-a531-436a-8161-67d2985155a4" containerName="maas-api"
Sep 07 16:31:41.055169 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.055110    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="46663950-a531-436a-8161-67d2985155a4" containerName="maas-api"
Sep 07 16:31:41.055256 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.055244    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="46663950-a531-436a-8161-67d2985155a4" containerName="maas-api"
Sep 07 16:31:41.060361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.060339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.062331 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.062306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1e4cf2b6-serving-cert\""
Sep 07 16:31:41.067242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.067208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:31:41.138342 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.138306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.138342 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.138344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.138551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.138365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.138551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.138396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqxdd\" (UniqueName: \"kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.239432 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.239396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqxdd\" (UniqueName: \"kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.239613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.239503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.239613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.239524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.239613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.239540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.239725 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:31:41.239645    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-serving-cert: secret "maas-api-e2e-migrate-1e4cf2b6-serving-cert" not found
Sep 07 16:31:41.239794 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:31:41.239725    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls podName:2bc884f6-fcf0-4bed-9f3b-dce593754d5f nodeName:}" failed. No retries permitted until 2026-09-07 16:31:41.739704175 +0000 UTC m=+740.383999462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls") pod "maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" (UID: "2bc884f6-fcf0-4bed-9f3b-dce593754d5f") : secret "maas-api-e2e-migrate-1e4cf2b6-serving-cert" not found
Sep 07 16:31:41.240260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.240233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.241970 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.241953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.247405 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.247384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqxdd\" (UniqueName: \"kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.744149 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.744111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.746570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.746550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") pod \"maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:41.973685 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:41.973650    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:42.255698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:42.255669    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:31:42.256824 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:31:42.256799    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc884f6_fcf0_4bed_9f3b_dce593754d5f.slice/crio-37e3875520460a237e6248ebed0a500a92f2aee6ccfdaa0d24e5945079e26a47 WatchSource:0}: Error finding container 37e3875520460a237e6248ebed0a500a92f2aee6ccfdaa0d24e5945079e26a47: Status 404 returned error can't find the container with id 37e3875520460a237e6248ebed0a500a92f2aee6ccfdaa0d24e5945079e26a47
Sep 07 16:31:43.248650 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:43.248608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" event={"ID":"2bc884f6-fcf0-4bed-9f3b-dce593754d5f","Type":"ContainerStarted","Data":"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"}
Sep 07 16:31:43.248904 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:43.248665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" event={"ID":"2bc884f6-fcf0-4bed-9f3b-dce593754d5f","Type":"ContainerStarted","Data":"37e3875520460a237e6248ebed0a500a92f2aee6ccfdaa0d24e5945079e26a47"}
Sep 07 16:31:43.248904 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:43.248703    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:43.267294 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:43.267234    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" podStartSLOduration=1.742354551 podStartE2EDuration="2.267217861s" totalImagesPullingTime="524.86331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:31:43.263986972 +0000 UTC m=+741.908282304" watchObservedRunningTime="2026-09-07 16:31:43.267217861 +0000 UTC m=+741.911513170"
Sep 07 16:31:43.338722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:43.338683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:31:49.261162 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:49.261130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:31:56.936551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:31:56.936509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:32:11.915160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:11.915121    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:32:11.915662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:11.915419    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" podUID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" containerName="maas-api" containerID="cri-o://b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920" gracePeriod=30
Sep 07 16:32:12.196626 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.196599    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:32:12.240794 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.240740    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") pod \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") "
Sep 07 16:32:12.240973 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.240826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs\") pod \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") "
Sep 07 16:32:12.240973 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.240892    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqxdd\" (UniqueName: \"kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd\") pod \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") "
Sep 07 16:32:12.240973 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.240938    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca\") pod \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\" (UID: \"2bc884f6-fcf0-4bed-9f3b-dce593754d5f\") "
Sep 07 16:32:12.241390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.241364    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bc884f6-fcf0-4bed-9f3b-dce593754d5f" (UID: "2bc884f6-fcf0-4bed-9f3b-dce593754d5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:32:12.243600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.243565    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd" (OuterVolumeSpecName: "kube-api-access-wqxdd") pod "2bc884f6-fcf0-4bed-9f3b-dce593754d5f" (UID: "2bc884f6-fcf0-4bed-9f3b-dce593754d5f"). InnerVolumeSpecName "kube-api-access-wqxdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:32:12.243698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.243646    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2bc884f6-fcf0-4bed-9f3b-dce593754d5f" (UID: "2bc884f6-fcf0-4bed-9f3b-dce593754d5f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:12.243941 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.243918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bc884f6-fcf0-4bed-9f3b-dce593754d5f" (UID: "2bc884f6-fcf0-4bed-9f3b-dce593754d5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:12.342460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.342423    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:12.342460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.342453    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqxdd\" (UniqueName: \"kubernetes.io/projected/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-kube-api-access-wqxdd\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:12.342460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.342465    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:12.342718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.342473    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bc884f6-fcf0-4bed-9f3b-dce593754d5f-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:12.434834 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.434703    2579 generic.go:356] "Generic (PLEG): container finished" podID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" containerID="b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920" exitCode=0
Sep 07 16:32:12.434834 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.434779    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"
Sep 07 16:32:12.434834 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.434793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" event={"ID":"2bc884f6-fcf0-4bed-9f3b-dce593754d5f","Type":"ContainerDied","Data":"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"}
Sep 07 16:32:12.435141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.434844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp" event={"ID":"2bc884f6-fcf0-4bed-9f3b-dce593754d5f","Type":"ContainerDied","Data":"37e3875520460a237e6248ebed0a500a92f2aee6ccfdaa0d24e5945079e26a47"}
Sep 07 16:32:12.435141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.434867    2579 scope.go:117] "RemoveContainer" containerID="b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"
Sep 07 16:32:12.445462 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.445443    2579 scope.go:117] "RemoveContainer" containerID="b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"
Sep 07 16:32:12.445795 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:32:12.445757    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920\": container with ID starting with b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920 not found: ID does not exist" containerID="b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"
Sep 07 16:32:12.445910 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.445800    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920"} err="failed to get container status \"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920\": rpc error: code = NotFound desc = could not find container \"b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920\": container with ID starting with b4a0a2ed36e6b65f4e8e1a7d8ab43e5f5682399d615db9a55333d6176313b920 not found: ID does not exist"
Sep 07 16:32:12.456108 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.456079    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:32:12.458822 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.458797    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1e4cf2b6-66fd4554bf-vf9cp"]
Sep 07 16:32:12.738229 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:12.738187    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:32:13.891002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:13.890960    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" path="/var/lib/kubelet/pods/2bc884f6-fcf0-4bed-9f3b-dce593754d5f/volumes"
Sep 07 16:32:17.137926 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.137890    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:17.139083 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.139057    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" containerName="maas-api"
Sep 07 16:32:17.139220 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.139105    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" containerName="maas-api"
Sep 07 16:32:17.139282 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.139257    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc884f6-fcf0-4bed-9f3b-dce593754d5f" containerName="maas-api"
Sep 07 16:32:17.142605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.142578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.145022 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.144995    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b1b87a00-serving-cert\""
Sep 07 16:32:17.154268 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.152724    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:17.186541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.186504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.186541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.186548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.186801 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.186698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.186801 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.186737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv5wf\" (UniqueName: \"kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.287459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.287420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.287654 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.287539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.287654 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:32:17.287557    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-serving-cert: secret "maas-api-e2e-ait-b1b87a00-serving-cert" not found
Sep 07 16:32:17.287654 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:32:17.287651    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls podName:2e12f10e-ddb7-4f07-b526-d614c5c594a1 nodeName:}" failed. No retries permitted until 2026-09-07 16:32:17.787628453 +0000 UTC m=+776.431923763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls") pod "maas-api-e2e-ait-b1b87a00-7867974d98-8z566" (UID: "2e12f10e-ddb7-4f07-b526-d614c5c594a1") : secret "maas-api-e2e-ait-b1b87a00-serving-cert" not found
Sep 07 16:32:17.287871 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.287721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.287871 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.287766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv5wf\" (UniqueName: \"kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.288506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.288482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.290493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.290468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.298023 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.297993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv5wf\" (UniqueName: \"kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.792071 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.792034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:17.794535 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:17.794506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") pod \"maas-api-e2e-ait-b1b87a00-7867974d98-8z566\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:18.059001 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:18.058908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:18.340138 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:18.340107    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:18.342501 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:32:18.342462    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e12f10e_ddb7_4f07_b526_d614c5c594a1.slice/crio-0b8f711c7a148ce94e458005b421c0aa3ae5f545ccf866b7c7320733ecb00a2b WatchSource:0}: Error finding container 0b8f711c7a148ce94e458005b421c0aa3ae5f545ccf866b7c7320733ecb00a2b: Status 404 returned error can't find the container with id 0b8f711c7a148ce94e458005b421c0aa3ae5f545ccf866b7c7320733ecb00a2b
Sep 07 16:32:18.474415 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:18.474377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" event={"ID":"2e12f10e-ddb7-4f07-b526-d614c5c594a1","Type":"ContainerStarted","Data":"0b8f711c7a148ce94e458005b421c0aa3ae5f545ccf866b7c7320733ecb00a2b"}
Sep 07 16:32:19.483011 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:19.482968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" event={"ID":"2e12f10e-ddb7-4f07-b526-d614c5c594a1","Type":"ContainerStarted","Data":"6773bf3d21d87e13c97228dd9fb317825837e41a455ebe63e312043c00a88ec7"}
Sep 07 16:32:19.483528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:19.483186    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:19.498938 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:19.498888    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" podStartSLOduration=2.133694289 podStartE2EDuration="2.498871747s" totalImagesPullingTime="365.177458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:32:19.496928144 +0000 UTC m=+778.141223465" watchObservedRunningTime="2026-09-07 16:32:19.498871747 +0000 UTC m=+778.143167056"
Sep 07 16:32:25.497075 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:25.497038    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:30.735302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:30.735261    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:32:47.538583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.538476    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:47.539073 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.538993    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" podUID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" containerName="maas-api" containerID="cri-o://6773bf3d21d87e13c97228dd9fb317825837e41a455ebe63e312043c00a88ec7" gracePeriod=30
Sep 07 16:32:47.665230 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.665197    2579 generic.go:356] "Generic (PLEG): container finished" podID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" containerID="6773bf3d21d87e13c97228dd9fb317825837e41a455ebe63e312043c00a88ec7" exitCode=0
Sep 07 16:32:47.665359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.665271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" event={"ID":"2e12f10e-ddb7-4f07-b526-d614c5c594a1","Type":"ContainerDied","Data":"6773bf3d21d87e13c97228dd9fb317825837e41a455ebe63e312043c00a88ec7"}
Sep 07 16:32:47.813911 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.813882    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:47.886899 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.886869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca\") pod \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") "
Sep 07 16:32:47.887105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.886938    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs\") pod \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") "
Sep 07 16:32:47.887105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.886993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") pod \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") "
Sep 07 16:32:47.887105 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.887022    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv5wf\" (UniqueName: \"kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf\") pod \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\" (UID: \"2e12f10e-ddb7-4f07-b526-d614c5c594a1\") "
Sep 07 16:32:47.887313 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.887285    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e12f10e-ddb7-4f07-b526-d614c5c594a1" (UID: "2e12f10e-ddb7-4f07-b526-d614c5c594a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:32:47.890092 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.890059    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf" (OuterVolumeSpecName: "kube-api-access-kv5wf") pod "2e12f10e-ddb7-4f07-b526-d614c5c594a1" (UID: "2e12f10e-ddb7-4f07-b526-d614c5c594a1"). InnerVolumeSpecName "kube-api-access-kv5wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:32:47.890720 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.890629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e12f10e-ddb7-4f07-b526-d614c5c594a1" (UID: "2e12f10e-ddb7-4f07-b526-d614c5c594a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:47.892911 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.892784    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2e12f10e-ddb7-4f07-b526-d614c5c594a1" (UID: "2e12f10e-ddb7-4f07-b526-d614c5c594a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:47.988032 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.987986    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:47.988032 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.988021    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e12f10e-ddb7-4f07-b526-d614c5c594a1-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:47.988032 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.988030    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv5wf\" (UniqueName: \"kubernetes.io/projected/2e12f10e-ddb7-4f07-b526-d614c5c594a1-kube-api-access-kv5wf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:47.988032 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:47.988039    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e12f10e-ddb7-4f07-b526-d614c5c594a1-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:48.673110 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:48.673070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566" event={"ID":"2e12f10e-ddb7-4f07-b526-d614c5c594a1","Type":"ContainerDied","Data":"0b8f711c7a148ce94e458005b421c0aa3ae5f545ccf866b7c7320733ecb00a2b"}
Sep 07 16:32:48.673110 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:48.673098    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"
Sep 07 16:32:48.673608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:48.673149    2579 scope.go:117] "RemoveContainer" containerID="6773bf3d21d87e13c97228dd9fb317825837e41a455ebe63e312043c00a88ec7"
Sep 07 16:32:48.695586 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:48.695548    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:48.698466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:48.698435    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1b87a00-7867974d98-8z566"]
Sep 07 16:32:49.889313 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:49.889281    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" path="/var/lib/kubelet/pods/2e12f10e-ddb7-4f07-b526-d614c5c594a1/volumes"
Sep 07 16:32:51.856726 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:51.856687    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:32:53.470727 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.470686    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:32:53.471266 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.471168    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" containerName="maas-api"
Sep 07 16:32:53.471266 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.471181    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" containerName="maas-api"
Sep 07 16:32:53.471340 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.471269    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e12f10e-ddb7-4f07-b526-d614c5c594a1" containerName="maas-api"
Sep 07 16:32:53.476015 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.475982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.480374 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.478324    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-39437224-serving-cert\""
Sep 07 16:32:53.486934 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.486900    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:32:53.541092 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.541059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.541304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.541124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.541304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.541210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.541304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.541260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pd9j\" (UniqueName: \"kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.642672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.642592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.642672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.642650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.642672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.642675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pd9j\" (UniqueName: \"kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.643052 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.642822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.643398 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.643366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.645917 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.645886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.646166 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.646147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.651584 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.651560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pd9j\" (UniqueName: \"kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j\") pod \"maas-api-e2e-derive-39437224-544bdcc948-b2mwh\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:53.809339 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:53.809245    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:54.093598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:54.093567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:32:54.097475 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:32:54.097440    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d3e996_51cf_4e2a_8da0_a940e7a4eb7c.slice/crio-cf9900ef46b6af2e649df73c6cefe169b93c9532ce6620b12a5753050b3a023c WatchSource:0}: Error finding container cf9900ef46b6af2e649df73c6cefe169b93c9532ce6620b12a5753050b3a023c: Status 404 returned error can't find the container with id cf9900ef46b6af2e649df73c6cefe169b93c9532ce6620b12a5753050b3a023c
Sep 07 16:32:54.718172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:54.718074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" event={"ID":"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c","Type":"ContainerStarted","Data":"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"}
Sep 07 16:32:54.718172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:54.718141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" event={"ID":"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c","Type":"ContainerStarted","Data":"cf9900ef46b6af2e649df73c6cefe169b93c9532ce6620b12a5753050b3a023c"}
Sep 07 16:32:54.718639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:54.718182    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:32:54.735379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:54.735324    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" podStartSLOduration=1.367917499 podStartE2EDuration="1.735307764s" totalImagesPullingTime="367.390265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:32:54.734387973 +0000 UTC m=+813.378683284" watchObservedRunningTime="2026-09-07 16:32:54.735307764 +0000 UTC m=+813.379603073"
Sep 07 16:32:57.031922 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.031886    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:32:57.032407 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.032134    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" podUID="2de2bdd5-9482-4647-88ac-bbf924189796" containerName="maas-api" containerID="cri-o://5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92" gracePeriod=30
Sep 07 16:32:57.302011 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.301979    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:32:57.380476 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca\") pod \"2de2bdd5-9482-4647-88ac-bbf924189796\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") "
Sep 07 16:32:57.380674 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380528    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs\") pod \"2de2bdd5-9482-4647-88ac-bbf924189796\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") "
Sep 07 16:32:57.380674 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380580    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r845c\" (UniqueName: \"kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c\") pod \"2de2bdd5-9482-4647-88ac-bbf924189796\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") "
Sep 07 16:32:57.380674 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380659    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls\") pod \"2de2bdd5-9482-4647-88ac-bbf924189796\" (UID: \"2de2bdd5-9482-4647-88ac-bbf924189796\") "
Sep 07 16:32:57.380894 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380872    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2de2bdd5-9482-4647-88ac-bbf924189796" (UID: "2de2bdd5-9482-4647-88ac-bbf924189796"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:32:57.381002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.380979    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de2bdd5-9482-4647-88ac-bbf924189796-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:57.383526 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.383486    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2de2bdd5-9482-4647-88ac-bbf924189796" (UID: "2de2bdd5-9482-4647-88ac-bbf924189796"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:57.383526 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.383508    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2de2bdd5-9482-4647-88ac-bbf924189796" (UID: "2de2bdd5-9482-4647-88ac-bbf924189796"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:32:57.383734 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.383633    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c" (OuterVolumeSpecName: "kube-api-access-r845c") pod "2de2bdd5-9482-4647-88ac-bbf924189796" (UID: "2de2bdd5-9482-4647-88ac-bbf924189796"). InnerVolumeSpecName "kube-api-access-r845c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:32:57.481864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.481827    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:57.481864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.481861    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de2bdd5-9482-4647-88ac-bbf924189796-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:57.481864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.481873    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r845c\" (UniqueName: \"kubernetes.io/projected/2de2bdd5-9482-4647-88ac-bbf924189796-kube-api-access-r845c\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:32:57.737150 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.737113    2579 generic.go:356] "Generic (PLEG): container finished" podID="2de2bdd5-9482-4647-88ac-bbf924189796" containerID="5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92" exitCode=0
Sep 07 16:32:57.737344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.737162    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"
Sep 07 16:32:57.737344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.737184    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" event={"ID":"2de2bdd5-9482-4647-88ac-bbf924189796","Type":"ContainerDied","Data":"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"}
Sep 07 16:32:57.737344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.737236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t" event={"ID":"2de2bdd5-9482-4647-88ac-bbf924189796","Type":"ContainerDied","Data":"930a266cd4820d3aabc262291aa424ed34c53d1af8edee6213f13f6e324a6d95"}
Sep 07 16:32:57.737344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.737260    2579 scope.go:117] "RemoveContainer" containerID="5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"
Sep 07 16:32:57.747705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.747683    2579 scope.go:117] "RemoveContainer" containerID="5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"
Sep 07 16:32:57.748048 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:32:57.748029    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92\": container with ID starting with 5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92 not found: ID does not exist" containerID="5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"
Sep 07 16:32:57.748099 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.748056    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92"} err="failed to get container status \"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92\": rpc error: code = NotFound desc = could not find container \"5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92\": container with ID starting with 5699bf456e83b660c583b5f4827e8e22ee192f3fe9977078a5e4acf34fad7f92 not found: ID does not exist"
Sep 07 16:32:57.757372 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.757343    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:32:57.762959 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.762933    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8a3403-57d6c658cc-m4d8t"]
Sep 07 16:32:57.888659 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:32:57.888624    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de2bdd5-9482-4647-88ac-bbf924189796" path="/var/lib/kubelet/pods/2de2bdd5-9482-4647-88ac-bbf924189796/volumes"
Sep 07 16:33:00.730008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:00.729979    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:33:08.581825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.581783    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:33:08.592179 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.584195    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de2bdd5-9482-4647-88ac-bbf924189796" containerName="maas-api"
Sep 07 16:33:08.592179 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.584223    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de2bdd5-9482-4647-88ac-bbf924189796" containerName="maas-api"
Sep 07 16:33:08.592179 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.584442    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de2bdd5-9482-4647-88ac-bbf924189796" containerName="maas-api"
Sep 07 16:33:08.592646 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.592612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.595194 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.595159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:33:08.596112 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.596085    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-cc0983-serving-cert\""
Sep 07 16:33:08.692053 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.692016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.692248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.692080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.692248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.692176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.692248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.692192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gk9r\" (UniqueName: \"kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.792895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.792860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.792895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.792898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gk9r\" (UniqueName: \"kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.793166 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.792933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.793166 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.792985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.793166 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:33:08.793094    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-serving-cert: secret "maas-api-e2e-explicit-cc0983-serving-cert" not found
Sep 07 16:33:08.793351 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:33:08.793175    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls podName:1bca247c-dc42-4c7d-938d-d3ee01dbfe6d nodeName:}" failed. No retries permitted until 2026-09-07 16:33:09.293155518 +0000 UTC m=+827.937450807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls") pod "maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" (UID: "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d") : secret "maas-api-e2e-explicit-cc0983-serving-cert" not found
Sep 07 16:33:08.793655 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.793636    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.795542 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.795518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:08.803644 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:08.803615    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gk9r\" (UniqueName: \"kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:09.298007 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.297970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:09.300662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.300635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") pod \"maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:09.508123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.508083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:09.630244 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.630206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:33:09.784740 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.784703    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:33:09.786539 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:33:09.786514    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bca247c_dc42_4c7d_938d_d3ee01dbfe6d.slice/crio-069789512cb0d3911adda7b1f47466199f98fb243c9cc60465d6eee22f541b69 WatchSource:0}: Error finding container 069789512cb0d3911adda7b1f47466199f98fb243c9cc60465d6eee22f541b69: Status 404 returned error can't find the container with id 069789512cb0d3911adda7b1f47466199f98fb243c9cc60465d6eee22f541b69
Sep 07 16:33:09.816101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:09.816011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" event={"ID":"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d","Type":"ContainerStarted","Data":"069789512cb0d3911adda7b1f47466199f98fb243c9cc60465d6eee22f541b69"}
Sep 07 16:33:10.826397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:10.826347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" event={"ID":"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d","Type":"ContainerStarted","Data":"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"}
Sep 07 16:33:10.826857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:10.826438    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:10.845866 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:10.845814    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" podStartSLOduration=2.385349126 podStartE2EDuration="2.845796914s" totalImagesPullingTime="460.447788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:33:10.843096409 +0000 UTC m=+829.487391750" watchObservedRunningTime="2026-09-07 16:33:10.845796914 +0000 UTC m=+829.490092223"
Sep 07 16:33:16.839235 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:16.839205    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:33:23.847386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:23.847350    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:33:23.847976 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:23.847594    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" podUID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" containerName="maas-api" containerID="cri-o://27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e" gracePeriod=30
Sep 07 16:33:24.116003 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.115973    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:33:24.249928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.249900    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs\") pod \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") "
Sep 07 16:33:24.250097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.249994    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pd9j\" (UniqueName: \"kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j\") pod \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") "
Sep 07 16:33:24.250160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.250140    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls\") pod \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") "
Sep 07 16:33:24.250247 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.250231    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca\") pod \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\" (UID: \"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c\") "
Sep 07 16:33:24.250583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.250551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" (UID: "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:33:24.252321 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.252292    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j" (OuterVolumeSpecName: "kube-api-access-4pd9j") pod "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" (UID: "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c"). InnerVolumeSpecName "kube-api-access-4pd9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:33:24.252422 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.252350    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" (UID: "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:33:24.252540 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.252522    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" (UID: "67d3e996-51cf-4e2a-8da0-a940e7a4eb7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:33:24.351499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.351397    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:33:24.351499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.351436    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:33:24.351499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.351450    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pd9j\" (UniqueName: \"kubernetes.io/projected/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-kube-api-access-4pd9j\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:33:24.351499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.351464    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:33:24.932883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.932850    2579 generic.go:356] "Generic (PLEG): container finished" podID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" containerID="27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e" exitCode=0
Sep 07 16:33:24.933431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.932894    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"
Sep 07 16:33:24.933431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.932934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" event={"ID":"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c","Type":"ContainerDied","Data":"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"}
Sep 07 16:33:24.933431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.932996    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh" event={"ID":"67d3e996-51cf-4e2a-8da0-a940e7a4eb7c","Type":"ContainerDied","Data":"cf9900ef46b6af2e649df73c6cefe169b93c9532ce6620b12a5753050b3a023c"}
Sep 07 16:33:24.933431 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.933033    2579 scope.go:117] "RemoveContainer" containerID="27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"
Sep 07 16:33:24.943565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.943547    2579 scope.go:117] "RemoveContainer" containerID="27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"
Sep 07 16:33:24.943907 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:33:24.943886    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e\": container with ID starting with 27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e not found: ID does not exist" containerID="27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"
Sep 07 16:33:24.944000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.943915    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e"} err="failed to get container status \"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e\": rpc error: code = NotFound desc = could not find container \"27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e\": container with ID starting with 27e76860c7df7170cd664b61d4d7a56146a4f56a2914e6a9689e601855d2c11e not found: ID does not exist"
Sep 07 16:33:24.960377 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.960342    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:33:24.970974 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:24.970944    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39437224-544bdcc948-b2mwh"]
Sep 07 16:33:25.888695 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:25.888648    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" path="/var/lib/kubelet/pods/67d3e996-51cf-4e2a-8da0-a940e7a4eb7c/volumes"
Sep 07 16:33:28.434471 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:28.434435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:33:33.849459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:33.849424    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:33:43.435739 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:33:43.435704    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:34:01.037656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:01.037616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:34:16.727367 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:16.727283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:34:21.872473 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:21.872446    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:34:21.874372 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:21.874347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:34:27.725845 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:27.725783    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:34:27.841233 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:27.841188    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:34:27.841444 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:27.841401    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" podUID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" containerName="maas-api" containerID="cri-o://b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422" gracePeriod=30
Sep 07 16:34:28.113244 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.113217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:34:28.157011 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.156980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs\") pod \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") "
Sep 07 16:34:28.157170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.157021    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") pod \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") "
Sep 07 16:34:28.157170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.157056    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca\") pod \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") "
Sep 07 16:34:28.157260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.157192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gk9r\" (UniqueName: \"kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r\") pod \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\" (UID: \"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d\") "
Sep 07 16:34:28.157425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.157405    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" (UID: "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:34:28.157627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.157609    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:34:28.159528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.159501    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r" (OuterVolumeSpecName: "kube-api-access-6gk9r") pod "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" (UID: "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d"). InnerVolumeSpecName "kube-api-access-6gk9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:34:28.159830 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.159805    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" (UID: "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:34:28.159912 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.159840    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" (UID: "1bca247c-dc42-4c7d-938d-d3ee01dbfe6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:34:28.258978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.258940    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:34:28.258978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.258970    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:34:28.258978 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.258979    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gk9r\" (UniqueName: \"kubernetes.io/projected/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d-kube-api-access-6gk9r\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:34:28.326527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.326434    2579 generic.go:356] "Generic (PLEG): container finished" podID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" containerID="b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422" exitCode=0
Sep 07 16:34:28.326527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.326484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" event={"ID":"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d","Type":"ContainerDied","Data":"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"}
Sep 07 16:34:28.326527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.326523    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn" event={"ID":"1bca247c-dc42-4c7d-938d-d3ee01dbfe6d","Type":"ContainerDied","Data":"069789512cb0d3911adda7b1f47466199f98fb243c9cc60465d6eee22f541b69"}
Sep 07 16:34:28.326843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.326544    2579 scope.go:117] "RemoveContainer" containerID="b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"
Sep 07 16:34:28.326843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.326483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"
Sep 07 16:34:28.337021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.337001    2579 scope.go:117] "RemoveContainer" containerID="b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"
Sep 07 16:34:28.337305 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:34:28.337281    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422\": container with ID starting with b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422 not found: ID does not exist" containerID="b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"
Sep 07 16:34:28.337374 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.337312    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422"} err="failed to get container status \"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422\": rpc error: code = NotFound desc = could not find container \"b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422\": container with ID starting with b13a88eb6682f52d9ead4a619715267ee359279aed64ff1d4fb653b2869ce422 not found: ID does not exist"
Sep 07 16:34:28.348589 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.348566    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:34:28.352764 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:28.352728    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc0983-588cc4f8f-2nwcn"]
Sep 07 16:34:29.890038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:29.890001    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" path="/var/lib/kubelet/pods/1bca247c-dc42-4c7d-938d-d3ee01dbfe6d/volumes"
Sep 07 16:34:38.730656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:34:38.730619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:35:02.641520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.641486    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:02.642097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642074    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" containerName="maas-api"
Sep 07 16:35:02.642097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642099    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" containerName="maas-api"
Sep 07 16:35:02.642264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642111    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" containerName="maas-api"
Sep 07 16:35:02.642264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642118    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" containerName="maas-api"
Sep 07 16:35:02.642649 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642629    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="67d3e996-51cf-4e2a-8da0-a940e7a4eb7c" containerName="maas-api"
Sep 07 16:35:02.642806 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.642727    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bca247c-dc42-4c7d-938d-d3ee01dbfe6d" containerName="maas-api"
Sep 07 16:35:02.646236 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.646215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.648563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.648538    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-65f07c0f-serving-cert\""
Sep 07 16:35:02.656080 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.656057    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:02.686530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.686505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.686688 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.686540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.686688 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.686569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.686688 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.686648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqbl7\" (UniqueName: \"kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.787853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.787819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.788025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.787864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.788025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.787894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.788025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.787935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqbl7\" (UniqueName: \"kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.788168 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:35:02.788078    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-serving-cert: secret "maas-api-e2e-mt-65f07c0f-serving-cert" not found
Sep 07 16:35:02.788206 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:35:02.788173    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls podName:b47db535-48b2-4a94-b1d2-4ae9a1953307 nodeName:}" failed. No retries permitted until 2026-09-07 16:35:03.288149806 +0000 UTC m=+941.932445115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls") pod "maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" (UID: "b47db535-48b2-4a94-b1d2-4ae9a1953307") : secret "maas-api-e2e-mt-65f07c0f-serving-cert" not found
Sep 07 16:35:02.788630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.788569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.790445 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.790424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:02.795930 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:02.795908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqbl7\" (UniqueName: \"kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:03.294006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:03.293966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:03.296498 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:03.296468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") pod \"maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:03.558715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:03.558628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:03.838875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:03.838809    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:03.841689 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:35:03.841653    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47db535_48b2_4a94_b1d2_4ae9a1953307.slice/crio-67f7074cf9b93d65da602f310acdc06086d2b17ae000fe2945dc813bc42e8c95 WatchSource:0}: Error finding container 67f7074cf9b93d65da602f310acdc06086d2b17ae000fe2945dc813bc42e8c95: Status 404 returned error can't find the container with id 67f7074cf9b93d65da602f310acdc06086d2b17ae000fe2945dc813bc42e8c95
Sep 07 16:35:03.843219 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:03.843200    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:35:04.553065 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:04.553031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" event={"ID":"b47db535-48b2-4a94-b1d2-4ae9a1953307","Type":"ContainerStarted","Data":"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"}
Sep 07 16:35:04.553315 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:04.553276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" event={"ID":"b47db535-48b2-4a94-b1d2-4ae9a1953307","Type":"ContainerStarted","Data":"67f7074cf9b93d65da602f310acdc06086d2b17ae000fe2945dc813bc42e8c95"}
Sep 07 16:35:04.553398 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:04.553320    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:04.570271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:04.570219    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" podStartSLOduration=1.9453980610000001 podStartE2EDuration="2.570199352s" totalImagesPullingTime="624.801291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:35:04.569616061 +0000 UTC m=+943.213911371" watchObservedRunningTime="2026-09-07 16:35:04.570199352 +0000 UTC m=+943.214494664"
Sep 07 16:35:10.564960 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:10.564929    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:11.041049 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:11.041017    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:35:19.697208 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:19.697167    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:35:19.697658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:19.697377    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" podUID="30abe198-977f-47ab-97a2-0f1019772840" containerName="maas-api" containerID="cri-o://72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95" gracePeriod=30
Sep 07 16:35:19.954475 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:19.954411    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:35:20.053803 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.053770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls\") pod \"30abe198-977f-47ab-97a2-0f1019772840\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") "
Sep 07 16:35:20.053984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.053827    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66jx4\" (UniqueName: \"kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4\") pod \"30abe198-977f-47ab-97a2-0f1019772840\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") "
Sep 07 16:35:20.053984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.053864    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs\") pod \"30abe198-977f-47ab-97a2-0f1019772840\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") "
Sep 07 16:35:20.053984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.053925    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca\") pod \"30abe198-977f-47ab-97a2-0f1019772840\" (UID: \"30abe198-977f-47ab-97a2-0f1019772840\") "
Sep 07 16:35:20.054408 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.054376    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30abe198-977f-47ab-97a2-0f1019772840" (UID: "30abe198-977f-47ab-97a2-0f1019772840"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:35:20.056196 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.056168    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4" (OuterVolumeSpecName: "kube-api-access-66jx4") pod "30abe198-977f-47ab-97a2-0f1019772840" (UID: "30abe198-977f-47ab-97a2-0f1019772840"). InnerVolumeSpecName "kube-api-access-66jx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:35:20.056317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.056193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30abe198-977f-47ab-97a2-0f1019772840" (UID: "30abe198-977f-47ab-97a2-0f1019772840"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:20.056317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.056275    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30abe198-977f-47ab-97a2-0f1019772840" (UID: "30abe198-977f-47ab-97a2-0f1019772840"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:20.154733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.154701    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:20.154733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.154734    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66jx4\" (UniqueName: \"kubernetes.io/projected/30abe198-977f-47ab-97a2-0f1019772840-kube-api-access-66jx4\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:20.154931 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.154776    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30abe198-977f-47ab-97a2-0f1019772840-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:20.154931 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.154788    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30abe198-977f-47ab-97a2-0f1019772840-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:20.652085 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.652051    2579 generic.go:356] "Generic (PLEG): container finished" podID="30abe198-977f-47ab-97a2-0f1019772840" containerID="72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95" exitCode=0
Sep 07 16:35:20.652085 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.652089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" event={"ID":"30abe198-977f-47ab-97a2-0f1019772840","Type":"ContainerDied","Data":"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"}
Sep 07 16:35:20.652296 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.652098    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"
Sep 07 16:35:20.652296 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.652130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx" event={"ID":"30abe198-977f-47ab-97a2-0f1019772840","Type":"ContainerDied","Data":"9e17aebb3b2c0de62d6ec97d18c03bb2987bdad40783f419c607f8d8c851d3a0"}
Sep 07 16:35:20.652296 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.652152    2579 scope.go:117] "RemoveContainer" containerID="72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"
Sep 07 16:35:20.662461 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.662445    2579 scope.go:117] "RemoveContainer" containerID="72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"
Sep 07 16:35:20.662735 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:35:20.662716    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95\": container with ID starting with 72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95 not found: ID does not exist" containerID="72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"
Sep 07 16:35:20.662819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.662764    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95"} err="failed to get container status \"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95\": rpc error: code = NotFound desc = could not find container \"72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95\": container with ID starting with 72b0230d4c5f691b55d2cf310ce3e031b55c74133c938c1a8f45f524b8086c95 not found: ID does not exist"
Sep 07 16:35:20.673346 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.673326    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:35:20.681981 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:20.681959    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-747d6d-57cc479b5d-cvvgx"]
Sep 07 16:35:21.890012 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:21.889978    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30abe198-977f-47ab-97a2-0f1019772840" path="/var/lib/kubelet/pods/30abe198-977f-47ab-97a2-0f1019772840/volumes"
Sep 07 16:35:23.239056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:23.239018    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:35:37.169146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.169111    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:35:37.169930 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.169886    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" podUID="c193c0fb-99a0-458c-a41d-0354bea3060a" containerName="maas-api" containerID="cri-o://f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0" gracePeriod=30
Sep 07 16:35:37.422816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.422759    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:35:37.516038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.516009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdtf9\" (UniqueName: \"kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9\") pod \"c193c0fb-99a0-458c-a41d-0354bea3060a\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") "
Sep 07 16:35:37.516215 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.516055    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") pod \"c193c0fb-99a0-458c-a41d-0354bea3060a\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") "
Sep 07 16:35:37.516215 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.516078    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca\") pod \"c193c0fb-99a0-458c-a41d-0354bea3060a\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") "
Sep 07 16:35:37.516215 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.516145    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs\") pod \"c193c0fb-99a0-458c-a41d-0354bea3060a\" (UID: \"c193c0fb-99a0-458c-a41d-0354bea3060a\") "
Sep 07 16:35:37.516465 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.516441    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c193c0fb-99a0-458c-a41d-0354bea3060a" (UID: "c193c0fb-99a0-458c-a41d-0354bea3060a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:35:37.518255 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.518220    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c193c0fb-99a0-458c-a41d-0354bea3060a" (UID: "c193c0fb-99a0-458c-a41d-0354bea3060a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:37.518341 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.518249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c193c0fb-99a0-458c-a41d-0354bea3060a" (UID: "c193c0fb-99a0-458c-a41d-0354bea3060a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:37.518341 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.518305    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9" (OuterVolumeSpecName: "kube-api-access-fdtf9") pod "c193c0fb-99a0-458c-a41d-0354bea3060a" (UID: "c193c0fb-99a0-458c-a41d-0354bea3060a"). InnerVolumeSpecName "kube-api-access-fdtf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:35:37.617308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.617270    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:37.617308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.617300    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdtf9\" (UniqueName: \"kubernetes.io/projected/c193c0fb-99a0-458c-a41d-0354bea3060a-kube-api-access-fdtf9\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:37.617308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.617314    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c193c0fb-99a0-458c-a41d-0354bea3060a-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:37.617531 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.617326    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c193c0fb-99a0-458c-a41d-0354bea3060a-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:37.765895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.765861    2579 generic.go:356] "Generic (PLEG): container finished" podID="c193c0fb-99a0-458c-a41d-0354bea3060a" containerID="f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0" exitCode=0
Sep 07 16:35:37.766038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.765904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"
Sep 07 16:35:37.766038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.765913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" event={"ID":"c193c0fb-99a0-458c-a41d-0354bea3060a","Type":"ContainerDied","Data":"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"}
Sep 07 16:35:37.766133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.766055    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js" event={"ID":"c193c0fb-99a0-458c-a41d-0354bea3060a","Type":"ContainerDied","Data":"77c3e349e55366c0eacb7000e0efef09a65800546710b88bc8fb3d3107b07cca"}
Sep 07 16:35:37.766133 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.766100    2579 scope.go:117] "RemoveContainer" containerID="f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"
Sep 07 16:35:37.776405 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.776387    2579 scope.go:117] "RemoveContainer" containerID="f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"
Sep 07 16:35:37.776677 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:35:37.776654    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0\": container with ID starting with f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0 not found: ID does not exist" containerID="f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"
Sep 07 16:35:37.776792 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.776688    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0"} err="failed to get container status \"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0\": rpc error: code = NotFound desc = could not find container \"f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0\": container with ID starting with f5d3086c0205b9094df9b9b1918bd689032350158ae128d6e6d22b41b81bf7f0 not found: ID does not exist"
Sep 07 16:35:37.787628 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.787603    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:35:37.790448 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.790427    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7d399-6c9984bdb8-xr5js"]
Sep 07 16:35:37.890122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:37.890094    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c193c0fb-99a0-458c-a41d-0354bea3060a" path="/var/lib/kubelet/pods/c193c0fb-99a0-458c-a41d-0354bea3060a/volumes"
Sep 07 16:35:40.532274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:40.532234    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:35:44.360991 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.360905    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:44.361548 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.361279    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" podUID="b47db535-48b2-4a94-b1d2-4ae9a1953307" containerName="maas-api" containerID="cri-o://e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b" gracePeriod=30
Sep 07 16:35:44.619662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.619601    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:44.683204 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683169    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca\") pod \"b47db535-48b2-4a94-b1d2-4ae9a1953307\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") "
Sep 07 16:35:44.683362 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683221    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") pod \"b47db535-48b2-4a94-b1d2-4ae9a1953307\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") "
Sep 07 16:35:44.683362 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs\") pod \"b47db535-48b2-4a94-b1d2-4ae9a1953307\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") "
Sep 07 16:35:44.683362 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqbl7\" (UniqueName: \"kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7\") pod \"b47db535-48b2-4a94-b1d2-4ae9a1953307\" (UID: \"b47db535-48b2-4a94-b1d2-4ae9a1953307\") "
Sep 07 16:35:44.683598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683573    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b47db535-48b2-4a94-b1d2-4ae9a1953307" (UID: "b47db535-48b2-4a94-b1d2-4ae9a1953307"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:35:44.683705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.683686    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b47db535-48b2-4a94-b1d2-4ae9a1953307-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:44.685510 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.685482    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b47db535-48b2-4a94-b1d2-4ae9a1953307" (UID: "b47db535-48b2-4a94-b1d2-4ae9a1953307"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:44.685630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.685499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7" (OuterVolumeSpecName: "kube-api-access-rqbl7") pod "b47db535-48b2-4a94-b1d2-4ae9a1953307" (UID: "b47db535-48b2-4a94-b1d2-4ae9a1953307"). InnerVolumeSpecName "kube-api-access-rqbl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:35:44.685630 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.685576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b47db535-48b2-4a94-b1d2-4ae9a1953307" (UID: "b47db535-48b2-4a94-b1d2-4ae9a1953307"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:35:44.784850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.784812    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:44.784850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.784840    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b47db535-48b2-4a94-b1d2-4ae9a1953307-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:44.784850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.784849    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqbl7\" (UniqueName: \"kubernetes.io/projected/b47db535-48b2-4a94-b1d2-4ae9a1953307-kube-api-access-rqbl7\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:35:44.812148 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.812119    2579 generic.go:356] "Generic (PLEG): container finished" podID="b47db535-48b2-4a94-b1d2-4ae9a1953307" containerID="e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b" exitCode=0
Sep 07 16:35:44.812302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.812171    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"
Sep 07 16:35:44.812302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.812203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" event={"ID":"b47db535-48b2-4a94-b1d2-4ae9a1953307","Type":"ContainerDied","Data":"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"}
Sep 07 16:35:44.812302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.812254    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf" event={"ID":"b47db535-48b2-4a94-b1d2-4ae9a1953307","Type":"ContainerDied","Data":"67f7074cf9b93d65da602f310acdc06086d2b17ae000fe2945dc813bc42e8c95"}
Sep 07 16:35:44.812302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.812276    2579 scope.go:117] "RemoveContainer" containerID="e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"
Sep 07 16:35:44.824538 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.824519    2579 scope.go:117] "RemoveContainer" containerID="e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"
Sep 07 16:35:44.824825 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:35:44.824803    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b\": container with ID starting with e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b not found: ID does not exist" containerID="e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"
Sep 07 16:35:44.824881 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.824833    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b"} err="failed to get container status \"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b\": rpc error: code = NotFound desc = could not find container \"e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b\": container with ID starting with e4725e46d9d0537920f569b4940ade87a6fa6ea8b9ffdec3fa27655fb6af619b not found: ID does not exist"
Sep 07 16:35:44.840503 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.840477    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:44.844453 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:44.844433    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65f07c0f-79f66dcdd5-54gpf"]
Sep 07 16:35:45.888636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:45.888601    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b47db535-48b2-4a94-b1d2-4ae9a1953307" path="/var/lib/kubelet/pods/b47db535-48b2-4a94-b1d2-4ae9a1953307/volumes"
Sep 07 16:35:47.945686 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:47.945643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:35:48.644123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644091    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:35:48.644565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644547    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47db535-48b2-4a94-b1d2-4ae9a1953307" containerName="maas-api"
Sep 07 16:35:48.644565 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644565    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47db535-48b2-4a94-b1d2-4ae9a1953307" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644575    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c193c0fb-99a0-458c-a41d-0354bea3060a" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644581    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c193c0fb-99a0-458c-a41d-0354bea3060a" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644587    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30abe198-977f-47ab-97a2-0f1019772840" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644592    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30abe198-977f-47ab-97a2-0f1019772840" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644681    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30abe198-977f-47ab-97a2-0f1019772840" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644694    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b47db535-48b2-4a94-b1d2-4ae9a1953307" containerName="maas-api"
Sep 07 16:35:48.644712 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.644702    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c193c0fb-99a0-458c-a41d-0354bea3060a" containerName="maas-api"
Sep 07 16:35:48.648170 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.648142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.650384 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.650359    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-8d59e2-serving-cert\""
Sep 07 16:35:48.658155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.658131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:35:48.720154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.720115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7d57\" (UniqueName: \"kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.720154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.720157    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.720382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.720177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.720382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.720303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.821094 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.821054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.821283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.821135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7d57\" (UniqueName: \"kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.821283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.821159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.821283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.821174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.821852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.821822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.823862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.823838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.823967 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.823845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.834374 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.834342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7d57\" (UniqueName: \"kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57\") pod \"maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:48.962180 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:48.962140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:49.240600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:49.240569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:35:49.242472 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:35:49.242440    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca529b6_de3d_4085_9d12_43b6de5a11fc.slice/crio-3f68711977ac086fdac588d86564a2d2039b6b26278752769ea041db11a5035b WatchSource:0}: Error finding container 3f68711977ac086fdac588d86564a2d2039b6b26278752769ea041db11a5035b: Status 404 returned error can't find the container with id 3f68711977ac086fdac588d86564a2d2039b6b26278752769ea041db11a5035b
Sep 07 16:35:49.843882 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:49.843785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" event={"ID":"eca529b6-de3d-4085-9d12-43b6de5a11fc","Type":"ContainerStarted","Data":"9d5d2afaf6415e44021ce8bcca37f84a47b7ac7c649a7028462cd695a8446b64"}
Sep 07 16:35:49.843882 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:49.843854    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:49.843882 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:49.843877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" event={"ID":"eca529b6-de3d-4085-9d12-43b6de5a11fc","Type":"ContainerStarted","Data":"3f68711977ac086fdac588d86564a2d2039b6b26278752769ea041db11a5035b"}
Sep 07 16:35:49.859954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:49.859893    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" podStartSLOduration=1.539695657 podStartE2EDuration="1.859872249s" totalImagesPullingTime="320.176592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:35:49.859165614 +0000 UTC m=+988.503460941" watchObservedRunningTime="2026-09-07 16:35:49.859872249 +0000 UTC m=+988.504167558"
Sep 07 16:35:55.855508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:55.855475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:35:58.235028 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:35:58.234990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:36:10.231144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.231109    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:36:10.237154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.237127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.239509 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.239484    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-bd6c5a-serving-cert\""
Sep 07 16:36:10.246555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.246525    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:36:10.425393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.425357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.425393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.425394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.425598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.425455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.425598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.425493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm2tn\" (UniqueName: \"kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.526258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.526172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.526258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.526215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.526450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.526264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.526450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.526294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm2tn\" (UniqueName: \"kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.526450 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:36:10.526395    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-serving-cert: secret "maas-api-e2e-shared-a-w2-bd6c5a-serving-cert" not found
Sep 07 16:36:10.526572 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:36:10.526475    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls podName:3c7e826c-6d75-4cd0-9a4e-f75652242ac0 nodeName:}" failed. No retries permitted until 2026-09-07 16:36:11.026455481 +0000 UTC m=+1009.670750784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls") pod "maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" (UID: "3c7e826c-6d75-4cd0-9a4e-f75652242ac0") : secret "maas-api-e2e-shared-a-w2-bd6c5a-serving-cert" not found
Sep 07 16:36:10.526949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.526921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.528821 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.528800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:10.539520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:10.539493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm2tn\" (UniqueName: \"kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:11.030684 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.030646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:11.033592 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.033559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:11.150081 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.150047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:11.427711 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.427683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:36:11.429198 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:36:11.429178    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7e826c_6d75_4cd0_9a4e_f75652242ac0.slice/crio-ff73e57cbe25b3d2f5520683d214a062d61d50aafcc2c2dbc9db57c2eba48d7d WatchSource:0}: Error finding container ff73e57cbe25b3d2f5520683d214a062d61d50aafcc2c2dbc9db57c2eba48d7d: Status 404 returned error can't find the container with id ff73e57cbe25b3d2f5520683d214a062d61d50aafcc2c2dbc9db57c2eba48d7d
Sep 07 16:36:11.998367 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.998330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" event={"ID":"3c7e826c-6d75-4cd0-9a4e-f75652242ac0","Type":"ContainerStarted","Data":"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"}
Sep 07 16:36:11.998520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.998380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" event={"ID":"3c7e826c-6d75-4cd0-9a4e-f75652242ac0","Type":"ContainerStarted","Data":"ff73e57cbe25b3d2f5520683d214a062d61d50aafcc2c2dbc9db57c2eba48d7d"}
Sep 07 16:36:11.998520 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:11.998441    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:12.014677 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:12.014625    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" podStartSLOduration=1.584422996 podStartE2EDuration="2.014607591s" totalImagesPullingTime="430.184595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:36:12.01434441 +0000 UTC m=+1010.658639744" watchObservedRunningTime="2026-09-07 16:36:12.014607591 +0000 UTC m=+1010.658902899"
Sep 07 16:36:13.542248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:13.542208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:36:17.055322 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.055122    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:36:17.059436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.059415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.061698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.061675    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-7aca0f-serving-cert\""
Sep 07 16:36:17.069695 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.069674    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:36:17.086366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.086343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.086551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.086387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.086551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.086527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.086693 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.086581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6gfl\" (UniqueName: \"kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.187169 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.187133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.187345 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.187179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6gfl\" (UniqueName: \"kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.187345 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.187200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.187345 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.187238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.188061 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.187963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.189703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.189681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.189846 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.189827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.200491 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.200468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6gfl\" (UniqueName: \"kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl\") pod \"maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.371989 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.371908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:17.653454 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:17.653429    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:36:17.655313 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:36:17.655286    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3592795_3ac0_4d16_82e7_37a8cfb5cd81.slice/crio-72e384794b6f46c9c323b11f9ee5bceb6a39ee81bf4e56a7dbcd243ec5324bc4 WatchSource:0}: Error finding container 72e384794b6f46c9c323b11f9ee5bceb6a39ee81bf4e56a7dbcd243ec5324bc4: Status 404 returned error can't find the container with id 72e384794b6f46c9c323b11f9ee5bceb6a39ee81bf4e56a7dbcd243ec5324bc4
Sep 07 16:36:18.009819 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:18.009789    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:36:18.035096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:18.035061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" event={"ID":"d3592795-3ac0-4d16-82e7-37a8cfb5cd81","Type":"ContainerStarted","Data":"72e384794b6f46c9c323b11f9ee5bceb6a39ee81bf4e56a7dbcd243ec5324bc4"}
Sep 07 16:36:19.042614 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:19.042573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" event={"ID":"d3592795-3ac0-4d16-82e7-37a8cfb5cd81","Type":"ContainerStarted","Data":"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"}
Sep 07 16:36:19.043095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:19.042629    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:19.060812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:19.060767    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" podStartSLOduration=1.688690614 podStartE2EDuration="2.060734156s" totalImagesPullingTime="372.043542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:36:19.057805507 +0000 UTC m=+1017.702100843" watchObservedRunningTime="2026-09-07 16:36:19.060734156 +0000 UTC m=+1017.705029464"
Sep 07 16:36:25.053286 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:25.053256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:36:27.045956 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:27.045921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:36:37.925296 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:37.925266    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:36:37.930549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:37.930521    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:37.932920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:37.932896    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4a9a21-serving-cert\""
Sep 07 16:36:37.940442 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:37.940417    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:36:38.080865 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.080829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.081050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.080887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6xsx\" (UniqueName: \"kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.081097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.081043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.081097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.081081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.182411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.182330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.182411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.182375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.182626 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.182451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.182626 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.182514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xsx\" (UniqueName: \"kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.183265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.183227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.185470 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.185444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.185628 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.185607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.190575 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.190554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6xsx\" (UniqueName: \"kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx\") pod \"maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.242901 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.242868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:38.523984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:38.523856    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:36:38.525638 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:36:38.525607    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db88c6a_b94b_43b7_a94a_6be8b37d4241.slice/crio-8afca7f4d4c28febe97d1427d08c601b820a78a1e6bb9b4959262b80ba9c2f78 WatchSource:0}: Error finding container 8afca7f4d4c28febe97d1427d08c601b820a78a1e6bb9b4959262b80ba9c2f78: Status 404 returned error can't find the container with id 8afca7f4d4c28febe97d1427d08c601b820a78a1e6bb9b4959262b80ba9c2f78
Sep 07 16:36:39.167969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:39.167873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" event={"ID":"7db88c6a-b94b-43b7-a94a-6be8b37d4241","Type":"ContainerStarted","Data":"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"}
Sep 07 16:36:39.167969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:39.167933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" event={"ID":"7db88c6a-b94b-43b7-a94a-6be8b37d4241","Type":"ContainerStarted","Data":"8afca7f4d4c28febe97d1427d08c601b820a78a1e6bb9b4959262b80ba9c2f78"}
Sep 07 16:36:39.168435 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:39.168037    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:39.187722 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:39.187668    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" podStartSLOduration=1.897836506 podStartE2EDuration="2.187652413s" totalImagesPullingTime="289.815907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:36:39.186446316 +0000 UTC m=+1037.830741648" watchObservedRunningTime="2026-09-07 16:36:39.187652413 +0000 UTC m=+1037.831947721"
Sep 07 16:36:42.840689 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:42.840653    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:36:45.179957 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:45.179927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:36:52.645106 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:36:52.645069    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:37:02.222616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.222583    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:02.227031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.227014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:02.229241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.229220    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9f7r\""
Sep 07 16:37:02.230967 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.230944    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:02.291531 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.291507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6zbt\" (UniqueName: \"kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt\") pod \"test-curl-7784-139799581513168\" (UID: \"5e56e6b5-3534-403b-9dc8-07e7b43442ca\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:02.393035 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.393003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6zbt\" (UniqueName: \"kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt\") pod \"test-curl-7784-139799581513168\" (UID: \"5e56e6b5-3534-403b-9dc8-07e7b43442ca\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:02.400534 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.400504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6zbt\" (UniqueName: \"kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt\") pod \"test-curl-7784-139799581513168\" (UID: \"5e56e6b5-3534-403b-9dc8-07e7b43442ca\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:02.538730 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.538647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:02.816371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:02.816344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:02.819322 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:37:02.819288    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e56e6b5_3534_403b_9dc8_07e7b43442ca.slice/crio-3ce3d33eeb261336b588e8193c491530ff47c797c4a15a2db16ab6cb73b5b117 WatchSource:0}: Error finding container 3ce3d33eeb261336b588e8193c491530ff47c797c4a15a2db16ab6cb73b5b117: Status 404 returned error can't find the container with id 3ce3d33eeb261336b588e8193c491530ff47c797c4a15a2db16ab6cb73b5b117
Sep 07 16:37:03.319426 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:03.319394    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513168" event={"ID":"5e56e6b5-3534-403b-9dc8-07e7b43442ca","Type":"ContainerStarted","Data":"3ce3d33eeb261336b588e8193c491530ff47c797c4a15a2db16ab6cb73b5b117"}
Sep 07 16:37:04.327036 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:04.326997    2579 generic.go:356] "Generic (PLEG): container finished" podID="5e56e6b5-3534-403b-9dc8-07e7b43442ca" containerID="18b9c1eee415e362f26788c17967b12bdb6dc07cf2a271b93d1ce2499a9d3ab9" exitCode=0
Sep 07 16:37:04.327397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:04.327077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513168" event={"ID":"5e56e6b5-3534-403b-9dc8-07e7b43442ca","Type":"ContainerDied","Data":"18b9c1eee415e362f26788c17967b12bdb6dc07cf2a271b93d1ce2499a9d3ab9"}
Sep 07 16:37:05.475335 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.475305    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:05.509529 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.509501    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7784-139799581513168_5e56e6b5-3534-403b-9dc8-07e7b43442ca/test-curl-7784-139799581513168/0.log"
Sep 07 16:37:05.517401 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.517375    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6zbt\" (UniqueName: \"kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt\") pod \"5e56e6b5-3534-403b-9dc8-07e7b43442ca\" (UID: \"5e56e6b5-3534-403b-9dc8-07e7b43442ca\") "
Sep 07 16:37:05.519518 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.519489    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt" (OuterVolumeSpecName: "kube-api-access-v6zbt") pod "5e56e6b5-3534-403b-9dc8-07e7b43442ca" (UID: "5e56e6b5-3534-403b-9dc8-07e7b43442ca"). InnerVolumeSpecName "kube-api-access-v6zbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:37:05.531031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.531011    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:05.533530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.533511    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:05.618330 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.618269    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6zbt\" (UniqueName: \"kubernetes.io/projected/5e56e6b5-3534-403b-9dc8-07e7b43442ca-kube-api-access-v6zbt\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:37:05.647659 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.647634    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:05.648094 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.648080    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e56e6b5-3534-403b-9dc8-07e7b43442ca" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:05.648145 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.648096    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e56e6b5-3534-403b-9dc8-07e7b43442ca" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:05.648194 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.648186    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e56e6b5-3534-403b-9dc8-07e7b43442ca" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:05.651308 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.651289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:05.657135 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.657112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:05.718847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.718820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs74x\" (UniqueName: \"kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x\") pod \"test-curl-7784-139799581513168\" (UID: \"f287af3f-f863-4cef-9d3b-16fb4b7f1c97\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:05.820201 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.820170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs74x\" (UniqueName: \"kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x\") pod \"test-curl-7784-139799581513168\" (UID: \"f287af3f-f863-4cef-9d3b-16fb4b7f1c97\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:05.827600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.827573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs74x\" (UniqueName: \"kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x\") pod \"test-curl-7784-139799581513168\" (UID: \"f287af3f-f863-4cef-9d3b-16fb4b7f1c97\") " pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:05.889110 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.889045    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e56e6b5-3534-403b-9dc8-07e7b43442ca" path="/var/lib/kubelet/pods/5e56e6b5-3534-403b-9dc8-07e7b43442ca/volumes"
Sep 07 16:37:05.962662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:05.962635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:06.232835 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:06.232811    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:06.234222 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:37:06.234195    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf287af3f_f863_4cef_9d3b_16fb4b7f1c97.slice/crio-5af49bcb3f58a262f719e16810c3e139be186d7a95457bf6c08e48d52ab614fa WatchSource:0}: Error finding container 5af49bcb3f58a262f719e16810c3e139be186d7a95457bf6c08e48d52ab614fa: Status 404 returned error can't find the container with id 5af49bcb3f58a262f719e16810c3e139be186d7a95457bf6c08e48d52ab614fa
Sep 07 16:37:06.342478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:06.342447    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:06.342661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:06.342505    2579 scope.go:117] "RemoveContainer" containerID="18b9c1eee415e362f26788c17967b12bdb6dc07cf2a271b93d1ce2499a9d3ab9"
Sep 07 16:37:06.343926 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:06.343815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513168" event={"ID":"f287af3f-f863-4cef-9d3b-16fb4b7f1c97","Type":"ContainerStarted","Data":"5af49bcb3f58a262f719e16810c3e139be186d7a95457bf6c08e48d52ab614fa"}
Sep 07 16:37:07.351298 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:07.351207    2579 generic.go:356] "Generic (PLEG): container finished" podID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" containerID="ffe38a74586675d3e3bc9e736e37bcfc10e4399be4cdab23960c3cc10d5faec3" exitCode=0
Sep 07 16:37:07.351298 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:07.351287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513168" event={"ID":"f287af3f-f863-4cef-9d3b-16fb4b7f1c97","Type":"ContainerDied","Data":"ffe38a74586675d3e3bc9e736e37bcfc10e4399be4cdab23960c3cc10d5faec3"}
Sep 07 16:37:08.489274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.489244    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:08.505676 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.505650    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7784-139799581513168_f287af3f-f863-4cef-9d3b-16fb4b7f1c97/test-curl-7784-139799581513168/0.log"
Sep 07 16:37:08.527882 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.527841    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:08.532020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.531995    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-139799581513168"]
Sep 07 16:37:08.545095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.545067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs74x\" (UniqueName: \"kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x\") pod \"f287af3f-f863-4cef-9d3b-16fb4b7f1c97\" (UID: \"f287af3f-f863-4cef-9d3b-16fb4b7f1c97\") "
Sep 07 16:37:08.547214 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.547188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x" (OuterVolumeSpecName: "kube-api-access-hs74x") pod "f287af3f-f863-4cef-9d3b-16fb4b7f1c97" (UID: "f287af3f-f863-4cef-9d3b-16fb4b7f1c97"). InnerVolumeSpecName "kube-api-access-hs74x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:37:08.638852 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.638778    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:08.639267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.639249    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:08.639309 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.639270    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:08.639356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.639347    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" containerName="test-curl-7784-139799581513168"
Sep 07 16:37:08.642467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.642443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:08.646908 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.646881    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs74x\" (UniqueName: \"kubernetes.io/projected/f287af3f-f863-4cef-9d3b-16fb4b7f1c97-kube-api-access-hs74x\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:37:08.656471 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.656436    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:08.748151 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.748115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9hj\" (UniqueName: \"kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj\") pod \"test-curl-7784-139799581513744\" (UID: \"f0551fec-0dac-4995-8872-3c15fadbd034\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:08.848909 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.848874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9hj\" (UniqueName: \"kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj\") pod \"test-curl-7784-139799581513744\" (UID: \"f0551fec-0dac-4995-8872-3c15fadbd034\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:08.856437 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.856415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9hj\" (UniqueName: \"kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj\") pod \"test-curl-7784-139799581513744\" (UID: \"f0551fec-0dac-4995-8872-3c15fadbd034\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:08.963828 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:08.963797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:09.238205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:09.238181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:09.239477 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:37:09.239453    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0551fec_0dac_4995_8872_3c15fadbd034.slice/crio-16eb7ad89aeb2c9bbc4985f17573eb21cca5263a7d4487806b6b5ccd0c3f8a31 WatchSource:0}: Error finding container 16eb7ad89aeb2c9bbc4985f17573eb21cca5263a7d4487806b6b5ccd0c3f8a31: Status 404 returned error can't find the container with id 16eb7ad89aeb2c9bbc4985f17573eb21cca5263a7d4487806b6b5ccd0c3f8a31
Sep 07 16:37:09.364305 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:09.364265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513744" event={"ID":"f0551fec-0dac-4995-8872-3c15fadbd034","Type":"ContainerStarted","Data":"16eb7ad89aeb2c9bbc4985f17573eb21cca5263a7d4487806b6b5ccd0c3f8a31"}
Sep 07 16:37:09.365606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:09.365584    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5af49bcb3f58a262f719e16810c3e139be186d7a95457bf6c08e48d52ab614fa"
Sep 07 16:37:09.365606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:09.365589    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:09.375426 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:37:09.375401    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7784-139799581513168\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" pod="opendatahub/test-curl-7784-139799581513168"
Sep 07 16:37:09.889321 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:09.889296    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f287af3f-f863-4cef-9d3b-16fb4b7f1c97" path="/var/lib/kubelet/pods/f287af3f-f863-4cef-9d3b-16fb4b7f1c97/volumes"
Sep 07 16:37:10.373360 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:10.373326    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0551fec-0dac-4995-8872-3c15fadbd034" containerID="976d6d062249b42296ffb503168afb2a078b238b6ecb8e0b27d73b43407f253a" exitCode=0
Sep 07 16:37:10.373532 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:10.373373    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513744" event={"ID":"f0551fec-0dac-4995-8872-3c15fadbd034","Type":"ContainerDied","Data":"976d6d062249b42296ffb503168afb2a078b238b6ecb8e0b27d73b43407f253a"}
Sep 07 16:37:11.520041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.520019    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:11.537257 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.537234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7784-139799581513744_f0551fec-0dac-4995-8872-3c15fadbd034/test-curl-7784-139799581513744/0.log"
Sep 07 16:37:11.557612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.557588    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:11.561915 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.561894    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:11.575406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.575386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p9hj\" (UniqueName: \"kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj\") pod \"f0551fec-0dac-4995-8872-3c15fadbd034\" (UID: \"f0551fec-0dac-4995-8872-3c15fadbd034\") "
Sep 07 16:37:11.577623 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.577596    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj" (OuterVolumeSpecName: "kube-api-access-6p9hj") pod "f0551fec-0dac-4995-8872-3c15fadbd034" (UID: "f0551fec-0dac-4995-8872-3c15fadbd034"). InnerVolumeSpecName "kube-api-access-6p9hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:37:11.668698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.668635    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:11.669096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.669082    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0551fec-0dac-4995-8872-3c15fadbd034" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:11.669141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.669098    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0551fec-0dac-4995-8872-3c15fadbd034" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:11.669182 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.669173    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0551fec-0dac-4995-8872-3c15fadbd034" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:11.672248 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.672229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:11.676087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.676064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:11.676404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.676380    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p9hj\" (UniqueName: \"kubernetes.io/projected/f0551fec-0dac-4995-8872-3c15fadbd034-kube-api-access-6p9hj\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:37:11.777262 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.777227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6jsv\" (UniqueName: \"kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv\") pod \"test-curl-7784-139799581513744\" (UID: \"a930b165-3cec-433b-b70e-96d601cd5675\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:11.878580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.878542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6jsv\" (UniqueName: \"kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv\") pod \"test-curl-7784-139799581513744\" (UID: \"a930b165-3cec-433b-b70e-96d601cd5675\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:11.887493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.887461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6jsv\" (UniqueName: \"kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv\") pod \"test-curl-7784-139799581513744\" (UID: \"a930b165-3cec-433b-b70e-96d601cd5675\") " pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:11.888681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.888661    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0551fec-0dac-4995-8872-3c15fadbd034" path="/var/lib/kubelet/pods/f0551fec-0dac-4995-8872-3c15fadbd034/volumes"
Sep 07 16:37:11.983287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:11.983255    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:12.252048 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:12.252018    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:12.253402 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:37:12.253376    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda930b165_3cec_433b_b70e_96d601cd5675.slice/crio-22fffc4803d8273c5dac0ef979ae3e213f946a50048f3aac63e305b9765f80f0 WatchSource:0}: Error finding container 22fffc4803d8273c5dac0ef979ae3e213f946a50048f3aac63e305b9765f80f0: Status 404 returned error can't find the container with id 22fffc4803d8273c5dac0ef979ae3e213f946a50048f3aac63e305b9765f80f0
Sep 07 16:37:12.387185 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:12.387146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513744" event={"ID":"a930b165-3cec-433b-b70e-96d601cd5675","Type":"ContainerStarted","Data":"22fffc4803d8273c5dac0ef979ae3e213f946a50048f3aac63e305b9765f80f0"}
Sep 07 16:37:12.388590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:12.388566    2579 scope.go:117] "RemoveContainer" containerID="976d6d062249b42296ffb503168afb2a078b238b6ecb8e0b27d73b43407f253a"
Sep 07 16:37:12.388590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:12.388587    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:13.396604 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:13.396565    2579 generic.go:356] "Generic (PLEG): container finished" podID="a930b165-3cec-433b-b70e-96d601cd5675" containerID="3a71e92fcfbc79669c37fabe38ed4186f6b3de5f0b52fe3cc4a7a1d3f394f93c" exitCode=0
Sep 07 16:37:13.397068 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:13.396652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-139799581513744" event={"ID":"a930b165-3cec-433b-b70e-96d601cd5675","Type":"ContainerDied","Data":"3a71e92fcfbc79669c37fabe38ed4186f6b3de5f0b52fe3cc4a7a1d3f394f93c"}
Sep 07 16:37:13.935890 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:13.935849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:37:14.544286 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.544257    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:14.562770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.562733    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7784-139799581513744_a930b165-3cec-433b-b70e-96d601cd5675/test-curl-7784-139799581513744/0.log"
Sep 07 16:37:14.586732 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.586699    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:14.590611 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.590587    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-139799581513744"]
Sep 07 16:37:14.602934 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.602910    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6jsv\" (UniqueName: \"kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv\") pod \"a930b165-3cec-433b-b70e-96d601cd5675\" (UID: \"a930b165-3cec-433b-b70e-96d601cd5675\") "
Sep 07 16:37:14.605045 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.605024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv" (OuterVolumeSpecName: "kube-api-access-r6jsv") pod "a930b165-3cec-433b-b70e-96d601cd5675" (UID: "a930b165-3cec-433b-b70e-96d601cd5675"). InnerVolumeSpecName "kube-api-access-r6jsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:37:14.704370 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:14.704346    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6jsv\" (UniqueName: \"kubernetes.io/projected/a930b165-3cec-433b-b70e-96d601cd5675-kube-api-access-r6jsv\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:37:15.417155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:15.417121    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:15.417155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:15.417145    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22fffc4803d8273c5dac0ef979ae3e213f946a50048f3aac63e305b9765f80f0"
Sep 07 16:37:15.427983 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:37:15.427954    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7784-139799581513744\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="a930b165-3cec-433b-b70e-96d601cd5675" pod="opendatahub/test-curl-7784-139799581513744"
Sep 07 16:37:15.890189 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:15.890154    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a930b165-3cec-433b-b70e-96d601cd5675" path="/var/lib/kubelet/pods/a930b165-3cec-433b-b70e-96d601cd5675/volumes"
Sep 07 16:37:19.351299 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.351257    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:37:19.351947 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.351926    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a930b165-3cec-433b-b70e-96d601cd5675" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:19.352037 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.351948    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a930b165-3cec-433b-b70e-96d601cd5675" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:19.352097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.352060    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a930b165-3cec-433b-b70e-96d601cd5675" containerName="test-curl-7784-139799581513744"
Sep 07 16:37:19.355404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.355383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.357532 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.357510    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2ff801c7-serving-cert\""
Sep 07 16:37:19.363639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.363585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:37:19.452486 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.452447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.452672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.452528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.452672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.452608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt57d\" (UniqueName: \"kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.452672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.452662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.553639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.553601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.553844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.553659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt57d\" (UniqueName: \"kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.553844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.553703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.553844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.553802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.554511 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.554482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.556484 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.556460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.556583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.556530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.561982 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.561954    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt57d\" (UniqueName: \"kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d\") pod \"maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.668506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.668408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:19.948850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:19.948821    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:37:19.950665 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:37:19.950640    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c01c86_39b1_46bc_9586_a4c215750f21.slice/crio-221e808f9b8f5b70474e3400b146e4ae76d028c241ae43b15c9f7e3acf8971f5 WatchSource:0}: Error finding container 221e808f9b8f5b70474e3400b146e4ae76d028c241ae43b15c9f7e3acf8971f5: Status 404 returned error can't find the container with id 221e808f9b8f5b70474e3400b146e4ae76d028c241ae43b15c9f7e3acf8971f5
Sep 07 16:37:20.452823 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:20.452784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" event={"ID":"00c01c86-39b1-46bc-9586-a4c215750f21","Type":"ContainerStarted","Data":"221e808f9b8f5b70474e3400b146e4ae76d028c241ae43b15c9f7e3acf8971f5"}
Sep 07 16:37:21.461197 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:21.461152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" event={"ID":"00c01c86-39b1-46bc-9586-a4c215750f21","Type":"ContainerStarted","Data":"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"}
Sep 07 16:37:21.461540 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:21.461230    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:21.478820 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:21.478770    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" podStartSLOduration=2.026229431 podStartE2EDuration="2.478736869s" totalImagesPullingTime="452.507438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:37:21.476316488 +0000 UTC m=+1080.120611799" watchObservedRunningTime="2026-09-07 16:37:21.478736869 +0000 UTC m=+1080.123032208"
Sep 07 16:37:21.991331 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:21.991299    2579 scope.go:117] "RemoveContainer" containerID="07488c3ac816e60dc0413b390bc81674f5f73e32528cb0ae40aba002ec3a8329"
Sep 07 16:37:22.002131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:22.002107    2579 scope.go:117] "RemoveContainer" containerID="e522bcad00d8a6d998d53796567069d26e8f8be06be9e949288922563ee05aba"
Sep 07 16:37:27.472450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:27.472417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:37:34.728021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:34.727986    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:37:51.639199 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:37:51.639154    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:38:09.729648 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:09.729610    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:38:26.630939 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:26.630903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:38:32.473287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.473251    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:38:32.473915 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.473590    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" podUID="00c01c86-39b1-46bc-9586-a4c215750f21" containerName="maas-api" containerID="cri-o://7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a" gracePeriod=30
Sep 07 16:38:32.731897 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.731836    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:38:32.847335 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.847307    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt57d\" (UniqueName: \"kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d\") pod \"00c01c86-39b1-46bc-9586-a4c215750f21\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") "
Sep 07 16:38:32.847512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.847361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca\") pod \"00c01c86-39b1-46bc-9586-a4c215750f21\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") "
Sep 07 16:38:32.847512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.847429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs\") pod \"00c01c86-39b1-46bc-9586-a4c215750f21\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") "
Sep 07 16:38:32.847600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.847555    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls\") pod \"00c01c86-39b1-46bc-9586-a4c215750f21\" (UID: \"00c01c86-39b1-46bc-9586-a4c215750f21\") "
Sep 07 16:38:32.847841 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.847811    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00c01c86-39b1-46bc-9586-a4c215750f21" (UID: "00c01c86-39b1-46bc-9586-a4c215750f21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:38:32.849775 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.849721    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "00c01c86-39b1-46bc-9586-a4c215750f21" (UID: "00c01c86-39b1-46bc-9586-a4c215750f21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:38:32.850379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.850353    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00c01c86-39b1-46bc-9586-a4c215750f21" (UID: "00c01c86-39b1-46bc-9586-a4c215750f21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:38:32.850493 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.850472    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d" (OuterVolumeSpecName: "kube-api-access-jt57d") pod "00c01c86-39b1-46bc-9586-a4c215750f21" (UID: "00c01c86-39b1-46bc-9586-a4c215750f21"). InnerVolumeSpecName "kube-api-access-jt57d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:38:32.899793 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.899727    2579 generic.go:356] "Generic (PLEG): container finished" podID="00c01c86-39b1-46bc-9586-a4c215750f21" containerID="7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a" exitCode=0
Sep 07 16:38:32.899793 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.899781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" event={"ID":"00c01c86-39b1-46bc-9586-a4c215750f21","Type":"ContainerDied","Data":"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"}
Sep 07 16:38:32.900044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.899807    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"
Sep 07 16:38:32.900044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.899850    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw" event={"ID":"00c01c86-39b1-46bc-9586-a4c215750f21","Type":"ContainerDied","Data":"221e808f9b8f5b70474e3400b146e4ae76d028c241ae43b15c9f7e3acf8971f5"}
Sep 07 16:38:32.900044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.899879    2579 scope.go:117] "RemoveContainer" containerID="7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"
Sep 07 16:38:32.910563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.910535    2579 scope.go:117] "RemoveContainer" containerID="7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"
Sep 07 16:38:32.910877 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:38:32.910856    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a\": container with ID starting with 7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a not found: ID does not exist" containerID="7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"
Sep 07 16:38:32.910946 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.910887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a"} err="failed to get container status \"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a\": rpc error: code = NotFound desc = could not find container \"7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a\": container with ID starting with 7579493c5b3edc3c9a4a24783308fa4c086c27a58467104f7f6bbea7b0e9981a not found: ID does not exist"
Sep 07 16:38:32.924344 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.924319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:38:32.928552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.928528    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ff801c7-6699fbb56f-xkcnw"]
Sep 07 16:38:32.948596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.948565    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:38:32.948596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.948597    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt57d\" (UniqueName: \"kubernetes.io/projected/00c01c86-39b1-46bc-9586-a4c215750f21-kube-api-access-jt57d\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:38:32.948729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.948611    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c01c86-39b1-46bc-9586-a4c215750f21-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:38:32.948729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:32.948624    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00c01c86-39b1-46bc-9586-a4c215750f21-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:38:33.892182 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:33.892138    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00c01c86-39b1-46bc-9586-a4c215750f21" path="/var/lib/kubelet/pods/00c01c86-39b1-46bc-9586-a4c215750f21/volumes"
Sep 07 16:38:43.433022 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:43.432940    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:38:59.328077 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:38:59.328038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:39:08.540758 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.540712    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7787-5c21fd"]
Sep 07 16:39:08.541218 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.541182    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00c01c86-39b1-46bc-9586-a4c215750f21" containerName="maas-api"
Sep 07 16:39:08.541218 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.541199    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="00c01c86-39b1-46bc-9586-a4c215750f21" containerName="maas-api"
Sep 07 16:39:08.541290 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.541282    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="00c01c86-39b1-46bc-9586-a4c215750f21" containerName="maas-api"
Sep 07 16:39:08.544627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.544608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:08.546874 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.546841    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9f7r\""
Sep 07 16:39:08.549692 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.549654    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-5c21fd"]
Sep 07 16:39:08.588928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.588899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c982h\" (UniqueName: \"kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h\") pod \"mt-curl-7787-5c21fd\" (UID: \"02b365c8-f4e2-4d23-ad97-3a052d134001\") " pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:08.689515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.689485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c982h\" (UniqueName: \"kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h\") pod \"mt-curl-7787-5c21fd\" (UID: \"02b365c8-f4e2-4d23-ad97-3a052d134001\") " pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:08.698328 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.698304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c982h\" (UniqueName: \"kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h\") pod \"mt-curl-7787-5c21fd\" (UID: \"02b365c8-f4e2-4d23-ad97-3a052d134001\") " pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:08.856611 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:08.856512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:09.126167 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:09.126139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-5c21fd"]
Sep 07 16:39:09.128105 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:39:09.128072    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02b365c8_f4e2_4d23_ad97_3a052d134001.slice/crio-7551967582b2bd8faa69990ea3bcca2b75f7901cbf3b3302ca86220a6f7c8b47 WatchSource:0}: Error finding container 7551967582b2bd8faa69990ea3bcca2b75f7901cbf3b3302ca86220a6f7c8b47: Status 404 returned error can't find the container with id 7551967582b2bd8faa69990ea3bcca2b75f7901cbf3b3302ca86220a6f7c8b47
Sep 07 16:39:10.137430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:10.137345    2579 generic.go:356] "Generic (PLEG): container finished" podID="02b365c8-f4e2-4d23-ad97-3a052d134001" containerID="897a2c42f52b1c25e24290f6483d0606706e0e41c1e4ccdce110a4aef40b229f" exitCode=0
Sep 07 16:39:10.137817 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:10.137431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-5c21fd" event={"ID":"02b365c8-f4e2-4d23-ad97-3a052d134001","Type":"ContainerDied","Data":"897a2c42f52b1c25e24290f6483d0606706e0e41c1e4ccdce110a4aef40b229f"}
Sep 07 16:39:10.137817 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:10.137487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-5c21fd" event={"ID":"02b365c8-f4e2-4d23-ad97-3a052d134001","Type":"ContainerStarted","Data":"7551967582b2bd8faa69990ea3bcca2b75f7901cbf3b3302ca86220a6f7c8b47"}
Sep 07 16:39:11.285290 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.285264    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:11.314089 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.314034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c982h\" (UniqueName: \"kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h\") pod \"02b365c8-f4e2-4d23-ad97-3a052d134001\" (UID: \"02b365c8-f4e2-4d23-ad97-3a052d134001\") "
Sep 07 16:39:11.315057 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.315028    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7787-5c21fd_02b365c8-f4e2-4d23-ad97-3a052d134001/mt-curl-7787-5c21fd/0.log"
Sep 07 16:39:11.316382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.316357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h" (OuterVolumeSpecName: "kube-api-access-c982h") pod "02b365c8-f4e2-4d23-ad97-3a052d134001" (UID: "02b365c8-f4e2-4d23-ad97-3a052d134001"). InnerVolumeSpecName "kube-api-access-c982h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:39:11.334959 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.334931    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7787-5c21fd"]
Sep 07 16:39:11.338472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.338451    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7787-5c21fd"]
Sep 07 16:39:11.415121 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.415063    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c982h\" (UniqueName: \"kubernetes.io/projected/02b365c8-f4e2-4d23-ad97-3a052d134001-kube-api-access-c982h\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:39:11.890035 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:11.890002    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b365c8-f4e2-4d23-ad97-3a052d134001" path="/var/lib/kubelet/pods/02b365c8-f4e2-4d23-ad97-3a052d134001/volumes"
Sep 07 16:39:12.151999 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:12.151913    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-5c21fd"
Sep 07 16:39:12.151999 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:12.151956    2579 scope.go:117] "RemoveContainer" containerID="897a2c42f52b1c25e24290f6483d0606706e0e41c1e4ccdce110a4aef40b229f"
Sep 07 16:39:15.329878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:15.329839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:39:20.733386 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.733343    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:39:20.733832 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.733804    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b365c8-f4e2-4d23-ad97-3a052d134001" containerName="mt-curl-7787-5c21fd"
Sep 07 16:39:20.733832 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.733816    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b365c8-f4e2-4d23-ad97-3a052d134001" containerName="mt-curl-7787-5c21fd"
Sep 07 16:39:20.733954 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.733892    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b365c8-f4e2-4d23-ad97-3a052d134001" containerName="mt-curl-7787-5c21fd"
Sep 07 16:39:20.737002 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.736980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.739861 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.739839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8d59e2\"/\"default-dockercfg-qcns6\""
Sep 07 16:39:20.740139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.739945    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8d59e2\"/\"openshift-service-ca.crt\""
Sep 07 16:39:20.740139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.739954    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8d59e2\"/\"sub-test-model-8d59e2-kserve-self-signed-certs\""
Sep 07 16:39:20.740139 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.739965    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8d59e2\"/\"kube-root-ca.crt\""
Sep 07 16:39:20.743937 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.743908    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:39:20.806194 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806158    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.806359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.806359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h89v\" (UniqueName: \"kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.806450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.806450 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.806552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.806516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.906964 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.906929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h89v\" (UniqueName: \"kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907434 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.907555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.907456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.909277 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.909258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.909547 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.909528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:20.914361 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:20.914339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h89v\" (UniqueName: \"kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v\") pod \"sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") " pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:21.048977 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:21.048897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:21.324213 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:21.324184    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:39:21.325682 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:39:21.325637    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e4b122c_1814_4d8c_a467_ec5429dbcb53.slice/crio-291cf3be8d10de67368bfb537e9a69099251105a40ac9df691479972bd1c7190 WatchSource:0}: Error finding container 291cf3be8d10de67368bfb537e9a69099251105a40ac9df691479972bd1c7190: Status 404 returned error can't find the container with id 291cf3be8d10de67368bfb537e9a69099251105a40ac9df691479972bd1c7190
Sep 07 16:39:21.916358 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:21.916325    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:39:21.918969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:21.918942    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:39:22.215353 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:22.215320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" event={"ID":"5e4b122c-1814-4d8c-a467-ec5429dbcb53","Type":"ContainerStarted","Data":"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"}
Sep 07 16:39:22.215535 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:22.215360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" event={"ID":"5e4b122c-1814-4d8c-a467-ec5429dbcb53","Type":"ContainerStarted","Data":"291cf3be8d10de67368bfb537e9a69099251105a40ac9df691479972bd1c7190"}
Sep 07 16:39:22.231260 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:22.231215    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" podStartSLOduration=2.23120037 podStartE2EDuration="2.23120037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:39:22.230125927 +0000 UTC m=+1200.874421232" watchObservedRunningTime="2026-09-07 16:39:22.23120037 +0000 UTC m=+1200.875495679"
Sep 07 16:39:31.049416 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:31.049381    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:31.049416 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:31.049422    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:31.061726 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:31.061700    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:31.280833 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:31.280808    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:39:33.239098 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:33.239065    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:39:43.941090 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:43.941056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:39:56.031436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:39:56.031398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:40:07.627373 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:07.627338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:40:20.039480 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:20.039402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:40:21.176799 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.176739    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:40:21.180720 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.180693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.182936 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.182907    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8ba4d4-serving-cert\""
Sep 07 16:40:21.193708 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.193681    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:40:21.303847 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.303816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.304008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.303871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwkr4\" (UniqueName: \"kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.304008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.303924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.304008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.303943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.404822 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.404791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.405008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.404854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwkr4\" (UniqueName: \"kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.405008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.404921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.405008 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.404947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.405008 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:21.404956    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-serving-cert: secret "maas-api-e2e-api-a-8ba4d4-serving-cert" not found
Sep 07 16:40:21.405174 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:21.405031    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls podName:7c5b27d9-c019-47ed-a155-be1ce72a0154 nodeName:}" failed. No retries permitted until 2026-09-07 16:40:21.905008028 +0000 UTC m=+1260.549303344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls") pod "maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" (UID: "7c5b27d9-c019-47ed-a155-be1ce72a0154") : secret "maas-api-e2e-api-a-8ba4d4-serving-cert" not found
Sep 07 16:40:21.405628 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.405592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.407566 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.407546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.412887 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.412869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwkr4\" (UniqueName: \"kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.909210 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.909170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:21.911911 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:21.911888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") pod \"maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:22.096218 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:22.096181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:22.378781 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:22.378728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:40:22.380480 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:40:22.380438    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5b27d9_c019_47ed_a155_be1ce72a0154.slice/crio-5d35e26f0a142332198fd4b89361b3c66f215b34258f335f135f9db70418411a WatchSource:0}: Error finding container 5d35e26f0a142332198fd4b89361b3c66f215b34258f335f135f9db70418411a: Status 404 returned error can't find the container with id 5d35e26f0a142332198fd4b89361b3c66f215b34258f335f135f9db70418411a
Sep 07 16:40:22.382270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:22.382248    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:40:22.607531 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:22.607490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" event={"ID":"7c5b27d9-c019-47ed-a155-be1ce72a0154","Type":"ContainerStarted","Data":"5d35e26f0a142332198fd4b89361b3c66f215b34258f335f135f9db70418411a"}
Sep 07 16:40:23.622406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.622359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" event={"ID":"7c5b27d9-c019-47ed-a155-be1ce72a0154","Type":"ContainerStarted","Data":"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"}
Sep 07 16:40:23.622872 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.622532    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:23.638924 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.638876    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" podStartSLOduration=2.097997783 podStartE2EDuration="2.638863638s" totalImagesPullingTime="540.865855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:40:23.637155853 +0000 UTC m=+1262.281451197" watchObservedRunningTime="2026-09-07 16:40:23.638863638 +0000 UTC m=+1262.283158946"
Sep 07 16:40:23.904900 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.904809    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7787-9751d3"]
Sep 07 16:40:23.908844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.908820    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:23.910858 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.910834    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9f7r\""
Sep 07 16:40:23.912140 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.912114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-9751d3"]
Sep 07 16:40:23.926359 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:23.926333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lvhj\" (UniqueName: \"kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj\") pod \"mt-curl-7787-9751d3\" (UID: \"164a893d-4358-4409-8de2-b8e3cec604ad\") " pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:24.027021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:24.026993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lvhj\" (UniqueName: \"kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj\") pod \"mt-curl-7787-9751d3\" (UID: \"164a893d-4358-4409-8de2-b8e3cec604ad\") " pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:24.034541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:24.034516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lvhj\" (UniqueName: \"kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj\") pod \"mt-curl-7787-9751d3\" (UID: \"164a893d-4358-4409-8de2-b8e3cec604ad\") " pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:24.220013 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:24.219986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:24.492969 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:24.492936    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-9751d3"]
Sep 07 16:40:24.494464 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:40:24.494437    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod164a893d_4358_4409_8de2_b8e3cec604ad.slice/crio-a46d49024a0c11f4f2fb21e1e6d0ed881e19a19621f54f13d0e247c830230743 WatchSource:0}: Error finding container a46d49024a0c11f4f2fb21e1e6d0ed881e19a19621f54f13d0e247c830230743: Status 404 returned error can't find the container with id a46d49024a0c11f4f2fb21e1e6d0ed881e19a19621f54f13d0e247c830230743
Sep 07 16:40:24.629929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:24.629896    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-9751d3" event={"ID":"164a893d-4358-4409-8de2-b8e3cec604ad","Type":"ContainerStarted","Data":"a46d49024a0c11f4f2fb21e1e6d0ed881e19a19621f54f13d0e247c830230743"}
Sep 07 16:40:25.637682 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:25.637641    2579 generic.go:356] "Generic (PLEG): container finished" podID="164a893d-4358-4409-8de2-b8e3cec604ad" containerID="66ba4f5d24fbd3fc717b1755c83d2b1804c341209d403c4da5164dbcea4732ab" exitCode=0
Sep 07 16:40:25.638102 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:25.637732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-9751d3" event={"ID":"164a893d-4358-4409-8de2-b8e3cec604ad","Type":"ContainerDied","Data":"66ba4f5d24fbd3fc717b1755c83d2b1804c341209d403c4da5164dbcea4732ab"}
Sep 07 16:40:26.788596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.788568    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:26.807135 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.807100    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7787-9751d3_164a893d-4358-4409-8de2-b8e3cec604ad/mt-curl-7787-9751d3/0.log"
Sep 07 16:40:26.826137 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.826097    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7787-9751d3"]
Sep 07 16:40:26.829580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.829556    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7787-9751d3"]
Sep 07 16:40:26.853382 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.853361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lvhj\" (UniqueName: \"kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj\") pod \"164a893d-4358-4409-8de2-b8e3cec604ad\" (UID: \"164a893d-4358-4409-8de2-b8e3cec604ad\") "
Sep 07 16:40:26.855577 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.855553    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj" (OuterVolumeSpecName: "kube-api-access-5lvhj") pod "164a893d-4358-4409-8de2-b8e3cec604ad" (UID: "164a893d-4358-4409-8de2-b8e3cec604ad"). InnerVolumeSpecName "kube-api-access-5lvhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:40:26.943316 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.943246    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7787-8301f9"]
Sep 07 16:40:26.943672 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.943660    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164a893d-4358-4409-8de2-b8e3cec604ad" containerName="mt-curl-7787-9751d3"
Sep 07 16:40:26.943714 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.943674    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="164a893d-4358-4409-8de2-b8e3cec604ad" containerName="mt-curl-7787-9751d3"
Sep 07 16:40:26.943772 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.943762    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="164a893d-4358-4409-8de2-b8e3cec604ad" containerName="mt-curl-7787-9751d3"
Sep 07 16:40:26.946735 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.946721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:26.952448 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.952423    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-8301f9"]
Sep 07 16:40:26.955156 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:26.955016    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lvhj\" (UniqueName: \"kubernetes.io/projected/164a893d-4358-4409-8de2-b8e3cec604ad-kube-api-access-5lvhj\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:27.056021 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.055993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksxjt\" (UniqueName: \"kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt\") pod \"mt-curl-7787-8301f9\" (UID: \"e10b676b-f074-488b-b1df-808f25eed600\") " pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:27.156853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.156825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksxjt\" (UniqueName: \"kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt\") pod \"mt-curl-7787-8301f9\" (UID: \"e10b676b-f074-488b-b1df-808f25eed600\") " pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:27.164728 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.164706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksxjt\" (UniqueName: \"kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt\") pod \"mt-curl-7787-8301f9\" (UID: \"e10b676b-f074-488b-b1df-808f25eed600\") " pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:27.257715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.257685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:27.545522 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.545498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7787-8301f9"]
Sep 07 16:40:27.547175 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:40:27.547154    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode10b676b_f074_488b_b1df_808f25eed600.slice/crio-f61b58b87e176cbda8ca7c7503277890426da899e83a77e90a4b46b0f56dbf53 WatchSource:0}: Error finding container f61b58b87e176cbda8ca7c7503277890426da899e83a77e90a4b46b0f56dbf53: Status 404 returned error can't find the container with id f61b58b87e176cbda8ca7c7503277890426da899e83a77e90a4b46b0f56dbf53
Sep 07 16:40:27.651006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.650975    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:27.651117 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.651003    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a46d49024a0c11f4f2fb21e1e6d0ed881e19a19621f54f13d0e247c830230743"
Sep 07 16:40:27.652205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.652182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-8301f9" event={"ID":"e10b676b-f074-488b-b1df-808f25eed600","Type":"ContainerStarted","Data":"f61b58b87e176cbda8ca7c7503277890426da899e83a77e90a4b46b0f56dbf53"}
Sep 07 16:40:27.660653 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:27.660623    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7787-9751d3\" is forbidden: User \"system:node:ip-10-0-135-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-154.ec2.internal' and this object" podUID="164a893d-4358-4409-8de2-b8e3cec604ad" pod="opendatahub/mt-curl-7787-9751d3"
Sep 07 16:40:27.890487 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:27.890457    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="164a893d-4358-4409-8de2-b8e3cec604ad" path="/var/lib/kubelet/pods/164a893d-4358-4409-8de2-b8e3cec604ad/volumes"
Sep 07 16:40:28.659850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:28.659817    2579 generic.go:356] "Generic (PLEG): container finished" podID="e10b676b-f074-488b-b1df-808f25eed600" containerID="4994a8833177b98f16ed58c0432690dacd15ede5780e72426cc7f0a159d202a2" exitCode=0
Sep 07 16:40:28.660029 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:28.659907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7787-8301f9" event={"ID":"e10b676b-f074-488b-b1df-808f25eed600","Type":"ContainerDied","Data":"4994a8833177b98f16ed58c0432690dacd15ede5780e72426cc7f0a159d202a2"}
Sep 07 16:40:29.635306 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.635276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:40:29.799228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.799206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:29.816605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.816579    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7787-8301f9_e10b676b-f074-488b-b1df-808f25eed600/mt-curl-7787-8301f9/0.log"
Sep 07 16:40:29.837414 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.837383    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7787-8301f9"]
Sep 07 16:40:29.839857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.839835    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7787-8301f9"]
Sep 07 16:40:29.881056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.881032    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksxjt\" (UniqueName: \"kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt\") pod \"e10b676b-f074-488b-b1df-808f25eed600\" (UID: \"e10b676b-f074-488b-b1df-808f25eed600\") "
Sep 07 16:40:29.883317 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.883286    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt" (OuterVolumeSpecName: "kube-api-access-ksxjt") pod "e10b676b-f074-488b-b1df-808f25eed600" (UID: "e10b676b-f074-488b-b1df-808f25eed600"). InnerVolumeSpecName "kube-api-access-ksxjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:40:29.893254 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.893216    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e10b676b-f074-488b-b1df-808f25eed600" path="/var/lib/kubelet/pods/e10b676b-f074-488b-b1df-808f25eed600/volumes"
Sep 07 16:40:29.982121 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:29.982079    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksxjt\" (UniqueName: \"kubernetes.io/projected/e10b676b-f074-488b-b1df-808f25eed600-kube-api-access-ksxjt\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:30.675809 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:30.675780    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7787-8301f9"
Sep 07 16:40:30.675809 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:30.675809    2579 scope.go:117] "RemoveContainer" containerID="4994a8833177b98f16ed58c0432690dacd15ede5780e72426cc7f0a159d202a2"
Sep 07 16:40:31.848220 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:31.848186    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:40:36.922929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:36.922891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:40:46.870014 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:46.869970    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" podUID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" containerName="main" containerID="cri-o://e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe" gracePeriod=45
Sep 07 16:40:47.123042 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.122982    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:40:47.245472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245621 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245621 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245515    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245621 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245606    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h89v\" (UniqueName: \"kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245622    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home\") pod \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\" (UID: \"5e4b122c-1814-4d8c-a467-ec5429dbcb53\") "
Sep 07 16:40:47.245770 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:40:47.246000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245954    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:40:47.246000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245962    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home" (OuterVolumeSpecName: "home") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:40:47.246000 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.245980    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-model-cache\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.247853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.247828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm" (OuterVolumeSpecName: "dshm") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:40:47.247939 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.247889    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:40:47.247939 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.247909    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v" (OuterVolumeSpecName: "kube-api-access-7h89v") pod "5e4b122c-1814-4d8c-a467-ec5429dbcb53" (UID: "5e4b122c-1814-4d8c-a467-ec5429dbcb53"). InnerVolumeSpecName "kube-api-access-7h89v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:40:47.346842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.346812    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h89v\" (UniqueName: \"kubernetes.io/projected/5e4b122c-1814-4d8c-a467-ec5429dbcb53-kube-api-access-7h89v\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.346842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.346838    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tls-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.346842 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.346847    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-home\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.347031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.346854    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-dshm\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.347031 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.346863    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4b122c-1814-4d8c-a467-ec5429dbcb53-tmp-dir\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:40:47.782259 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.782220    2579 generic.go:356] "Generic (PLEG): container finished" podID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" containerID="e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe" exitCode=0
Sep 07 16:40:47.782430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.782274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" event={"ID":"5e4b122c-1814-4d8c-a467-ec5429dbcb53","Type":"ContainerDied","Data":"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"}
Sep 07 16:40:47.782430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.782310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l" event={"ID":"5e4b122c-1814-4d8c-a467-ec5429dbcb53","Type":"ContainerDied","Data":"291cf3be8d10de67368bfb537e9a69099251105a40ac9df691479972bd1c7190"}
Sep 07 16:40:47.782430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.782333    2579 scope.go:117] "RemoveContainer" containerID="e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"
Sep 07 16:40:47.782430 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.782275    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"
Sep 07 16:40:47.793191 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.793173    2579 scope.go:117] "RemoveContainer" containerID="e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"
Sep 07 16:40:47.793473 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:47.793454    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe\": container with ID starting with e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe not found: ID does not exist" containerID="e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"
Sep 07 16:40:47.793553 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.793485    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe"} err="failed to get container status \"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe\": rpc error: code = NotFound desc = could not find container \"e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe\": container with ID starting with e1c65272d4c33a18a77580faad90dba268287a97cb5fcce06b87450f8651cabe not found: ID does not exist"
Sep 07 16:40:47.805718 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.805691    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:40:47.809676 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.809645    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8d59e2/sub-test-model-8d59e2-kserve-7b877c7f89-pbk5l"]
Sep 07 16:40:47.889390 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:47.889354    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" path="/var/lib/kubelet/pods/5e4b122c-1814-4d8c-a467-ec5429dbcb53/volumes"
Sep 07 16:40:50.335734 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.335704    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:40:50.336171 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336157    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" containerName="main"
Sep 07 16:40:50.336211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336174    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" containerName="main"
Sep 07 16:40:50.336211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336193    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10b676b-f074-488b-b1df-808f25eed600" containerName="mt-curl-7787-8301f9"
Sep 07 16:40:50.336211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336199    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10b676b-f074-488b-b1df-808f25eed600" containerName="mt-curl-7787-8301f9"
Sep 07 16:40:50.336334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336270    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10b676b-f074-488b-b1df-808f25eed600" containerName="mt-curl-7787-8301f9"
Sep 07 16:40:50.336334 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.336281    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e4b122c-1814-4d8c-a467-ec5429dbcb53" containerName="main"
Sep 07 16:40:50.343473 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.341488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.350294 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.350269    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d7b6a5-serving-cert\""
Sep 07 16:40:50.357551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.357520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:40:50.475078 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.475047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.475252 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.475100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.475252 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.475225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snrll\" (UniqueName: \"kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.475343 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.475293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.576772 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.576713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.576942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.576787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.576942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.576865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snrll\" (UniqueName: \"kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.576942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.576912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.577204 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:50.577183    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-serving-cert: secret "maas-api-e2e-api-b-d7b6a5-serving-cert" not found
Sep 07 16:40:50.577357 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:40:50.577342    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls podName:15b83ab3-b771-495b-9f3f-b656b564269f nodeName:}" failed. No retries permitted until 2026-09-07 16:40:51.077321662 +0000 UTC m=+1289.721616957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls") pod "maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" (UID: "15b83ab3-b771-495b-9f3f-b656b564269f") : secret "maas-api-e2e-api-b-d7b6a5-serving-cert" not found
Sep 07 16:40:50.577810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.577788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.579659 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.579633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:50.587723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:50.587669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snrll\" (UniqueName: \"kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:51.089383 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.089348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:51.091930 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.091909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") pod \"maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:51.262425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.262385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:51.543398 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.543370    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:40:51.544698 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:40:51.544668    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b83ab3_b771_495b_9f3f_b656b564269f.slice/crio-d40df8a5c25cb948644f4aa0104bd41579512f11c8043c3246b2b7d429073350 WatchSource:0}: Error finding container d40df8a5c25cb948644f4aa0104bd41579512f11c8043c3246b2b7d429073350: Status 404 returned error can't find the container with id d40df8a5c25cb948644f4aa0104bd41579512f11c8043c3246b2b7d429073350
Sep 07 16:40:51.777678 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.777639    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:40:51.809517 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:51.809424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" event={"ID":"15b83ab3-b771-495b-9f3f-b656b564269f","Type":"ContainerStarted","Data":"d40df8a5c25cb948644f4aa0104bd41579512f11c8043c3246b2b7d429073350"}
Sep 07 16:40:52.816854 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:52.816820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" event={"ID":"15b83ab3-b771-495b-9f3f-b656b564269f","Type":"ContainerStarted","Data":"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"}
Sep 07 16:40:52.817262 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:52.816966    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:40:52.833724 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:52.833671    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" podStartSLOduration=2.5069800840000003 podStartE2EDuration="2.833656048s" totalImagesPullingTime="326.675964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:40:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:40:52.832285334 +0000 UTC m=+1291.476580642" watchObservedRunningTime="2026-09-07 16:40:52.833656048 +0000 UTC m=+1291.477951426"
Sep 07 16:40:58.828267 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:40:58.828234    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:41:07.226839 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:07.226798    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:41:32.485468 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.485425    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:41:32.485974 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.485684    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" podUID="7c5b27d9-c019-47ed-a155-be1ce72a0154" containerName="maas-api" containerID="cri-o://59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c" gracePeriod=30
Sep 07 16:41:32.749246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.749184    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:41:32.865528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.865497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") pod \"7c5b27d9-c019-47ed-a155-be1ce72a0154\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") "
Sep 07 16:41:32.865696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.865545    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca\") pod \"7c5b27d9-c019-47ed-a155-be1ce72a0154\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") "
Sep 07 16:41:32.865696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.865594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwkr4\" (UniqueName: \"kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4\") pod \"7c5b27d9-c019-47ed-a155-be1ce72a0154\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") "
Sep 07 16:41:32.865696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.865654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs\") pod \"7c5b27d9-c019-47ed-a155-be1ce72a0154\" (UID: \"7c5b27d9-c019-47ed-a155-be1ce72a0154\") "
Sep 07 16:41:32.865998 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.865965    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c5b27d9-c019-47ed-a155-be1ce72a0154" (UID: "7c5b27d9-c019-47ed-a155-be1ce72a0154"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:41:32.867928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.867899    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c5b27d9-c019-47ed-a155-be1ce72a0154" (UID: "7c5b27d9-c019-47ed-a155-be1ce72a0154"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:41:32.867928 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.867913    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4" (OuterVolumeSpecName: "kube-api-access-rwkr4") pod "7c5b27d9-c019-47ed-a155-be1ce72a0154" (UID: "7c5b27d9-c019-47ed-a155-be1ce72a0154"). InnerVolumeSpecName "kube-api-access-rwkr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:41:32.868126 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.867924    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c5b27d9-c019-47ed-a155-be1ce72a0154" (UID: "7c5b27d9-c019-47ed-a155-be1ce72a0154"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:41:32.967606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.967572    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:32.967810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.967712    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c5b27d9-c019-47ed-a155-be1ce72a0154-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:32.967810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.967734    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5b27d9-c019-47ed-a155-be1ce72a0154-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:32.967810 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:32.967756    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwkr4\" (UniqueName: \"kubernetes.io/projected/7c5b27d9-c019-47ed-a155-be1ce72a0154-kube-api-access-rwkr4\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:33.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.063052    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c5b27d9-c019-47ed-a155-be1ce72a0154" containerID="59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c" exitCode=0
Sep 07 16:41:33.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.063099    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"
Sep 07 16:41:33.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.063105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" event={"ID":"7c5b27d9-c019-47ed-a155-be1ce72a0154","Type":"ContainerDied","Data":"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"}
Sep 07 16:41:33.063143 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.063147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k" event={"ID":"7c5b27d9-c019-47ed-a155-be1ce72a0154","Type":"ContainerDied","Data":"5d35e26f0a142332198fd4b89361b3c66f215b34258f335f135f9db70418411a"}
Sep 07 16:41:33.063436 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.063168    2579 scope.go:117] "RemoveContainer" containerID="59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"
Sep 07 16:41:33.074594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.074575    2579 scope.go:117] "RemoveContainer" containerID="59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"
Sep 07 16:41:33.074876 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:41:33.074858    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c\": container with ID starting with 59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c not found: ID does not exist" containerID="59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"
Sep 07 16:41:33.074953 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.074885    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c"} err="failed to get container status \"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c\": rpc error: code = NotFound desc = could not find container \"59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c\": container with ID starting with 59a7660a9452b356d024b56808ea2915e11ae5050a1c6f18ecf1537932452f2c not found: ID does not exist"
Sep 07 16:41:33.089281 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.089256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:41:33.093404 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.093379    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ba4d4-cfc5d6f5d-chd4k"]
Sep 07 16:41:33.888418 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:33.888387    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c5b27d9-c019-47ed-a155-be1ce72a0154" path="/var/lib/kubelet/pods/7c5b27d9-c019-47ed-a155-be1ce72a0154/volumes"
Sep 07 16:41:38.140123 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:38.140085    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:41:55.169967 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.169862    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:41:55.170523 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.170131    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" podUID="15b83ab3-b771-495b-9f3f-b656b564269f" containerName="maas-api" containerID="cri-o://8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e" gracePeriod=30
Sep 07 16:41:55.429700 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.429646    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:41:55.485489 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.485461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca\") pod \"15b83ab3-b771-495b-9f3f-b656b564269f\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") "
Sep 07 16:41:55.485654 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.485511    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs\") pod \"15b83ab3-b771-495b-9f3f-b656b564269f\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") "
Sep 07 16:41:55.485654 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.485555    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") pod \"15b83ab3-b771-495b-9f3f-b656b564269f\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") "
Sep 07 16:41:55.485654 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.485583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snrll\" (UniqueName: \"kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll\") pod \"15b83ab3-b771-495b-9f3f-b656b564269f\" (UID: \"15b83ab3-b771-495b-9f3f-b656b564269f\") "
Sep 07 16:41:55.485942 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.485905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15b83ab3-b771-495b-9f3f-b656b564269f" (UID: "15b83ab3-b771-495b-9f3f-b656b564269f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:41:55.488023 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.487997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15b83ab3-b771-495b-9f3f-b656b564269f" (UID: "15b83ab3-b771-495b-9f3f-b656b564269f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:41:55.488140 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.488000    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll" (OuterVolumeSpecName: "kube-api-access-snrll") pod "15b83ab3-b771-495b-9f3f-b656b564269f" (UID: "15b83ab3-b771-495b-9f3f-b656b564269f"). InnerVolumeSpecName "kube-api-access-snrll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:41:55.488140 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.488101    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15b83ab3-b771-495b-9f3f-b656b564269f" (UID: "15b83ab3-b771-495b-9f3f-b656b564269f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:41:55.586455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.586425    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:55.586455 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.586457    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b83ab3-b771-495b-9f3f-b656b564269f-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:55.586652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.586470    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snrll\" (UniqueName: \"kubernetes.io/projected/15b83ab3-b771-495b-9f3f-b656b564269f-kube-api-access-snrll\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:55.586652 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:55.586487    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b83ab3-b771-495b-9f3f-b656b564269f-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:41:56.208914 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.208882    2579 generic.go:356] "Generic (PLEG): container finished" podID="15b83ab3-b771-495b-9f3f-b656b564269f" containerID="8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e" exitCode=0
Sep 07 16:41:56.209356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.208940    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"
Sep 07 16:41:56.209356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.208969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" event={"ID":"15b83ab3-b771-495b-9f3f-b656b564269f","Type":"ContainerDied","Data":"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"}
Sep 07 16:41:56.209356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.209066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw" event={"ID":"15b83ab3-b771-495b-9f3f-b656b564269f","Type":"ContainerDied","Data":"d40df8a5c25cb948644f4aa0104bd41579512f11c8043c3246b2b7d429073350"}
Sep 07 16:41:56.209356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.209098    2579 scope.go:117] "RemoveContainer" containerID="8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"
Sep 07 16:41:56.219004 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.218986    2579 scope.go:117] "RemoveContainer" containerID="8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"
Sep 07 16:41:56.219279 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:41:56.219258    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e\": container with ID starting with 8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e not found: ID does not exist" containerID="8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"
Sep 07 16:41:56.219375 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.219284    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e"} err="failed to get container status \"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e\": rpc error: code = NotFound desc = could not find container \"8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e\": container with ID starting with 8fc28c8a293ddeee65b4eeadce26b6b0b9929a402075485d295fb207b13cbb7e not found: ID does not exist"
Sep 07 16:41:56.224291 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.224269    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:41:56.227984 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:56.227961    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d7b6a5-6df859fc67-h59rw"]
Sep 07 16:41:57.889100 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:57.889065    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15b83ab3-b771-495b-9f3f-b656b564269f" path="/var/lib/kubelet/pods/15b83ab3-b771-495b-9f3f-b656b564269f/volumes"
Sep 07 16:41:58.929506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:41:58.929463    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:42:12.426715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.426673    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:42:12.427287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.426920    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" podUID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" containerName="maas-api" containerID="cri-o://2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc" gracePeriod=30
Sep 07 16:42:12.685647 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.685580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:42:12.747371 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.747342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm2tn\" (UniqueName: \"kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn\") pod \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") "
Sep 07 16:42:12.747515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.747419    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") pod \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") "
Sep 07 16:42:12.747515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.747456    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs\") pod \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") "
Sep 07 16:42:12.747515 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.747491    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca\") pod \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\" (UID: \"3c7e826c-6d75-4cd0-9a4e-f75652242ac0\") "
Sep 07 16:42:12.747949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.747918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c7e826c-6d75-4cd0-9a4e-f75652242ac0" (UID: "3c7e826c-6d75-4cd0-9a4e-f75652242ac0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:42:12.749603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.749572    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn" (OuterVolumeSpecName: "kube-api-access-dm2tn") pod "3c7e826c-6d75-4cd0-9a4e-f75652242ac0" (UID: "3c7e826c-6d75-4cd0-9a4e-f75652242ac0"). InnerVolumeSpecName "kube-api-access-dm2tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:42:12.749603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.749581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c7e826c-6d75-4cd0-9a4e-f75652242ac0" (UID: "3c7e826c-6d75-4cd0-9a4e-f75652242ac0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:42:12.749603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.749595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c7e826c-6d75-4cd0-9a4e-f75652242ac0" (UID: "3c7e826c-6d75-4cd0-9a4e-f75652242ac0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:42:12.849049 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.849016    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:12.849049 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.849043    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:12.849049 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.849052    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:12.849274 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:12.849062    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm2tn\" (UniqueName: \"kubernetes.io/projected/3c7e826c-6d75-4cd0-9a4e-f75652242ac0-kube-api-access-dm2tn\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:13.316685 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.316642    2579 generic.go:356] "Generic (PLEG): container finished" podID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" containerID="2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc" exitCode=0
Sep 07 16:42:13.316907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.316693    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"
Sep 07 16:42:13.316907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.316718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" event={"ID":"3c7e826c-6d75-4cd0-9a4e-f75652242ac0","Type":"ContainerDied","Data":"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"}
Sep 07 16:42:13.316907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.316791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz" event={"ID":"3c7e826c-6d75-4cd0-9a4e-f75652242ac0","Type":"ContainerDied","Data":"ff73e57cbe25b3d2f5520683d214a062d61d50aafcc2c2dbc9db57c2eba48d7d"}
Sep 07 16:42:13.316907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.316817    2579 scope.go:117] "RemoveContainer" containerID="2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"
Sep 07 16:42:13.327360 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.327342    2579 scope.go:117] "RemoveContainer" containerID="2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"
Sep 07 16:42:13.327664 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:42:13.327631    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc\": container with ID starting with 2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc not found: ID does not exist" containerID="2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"
Sep 07 16:42:13.327708 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.327688    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc"} err="failed to get container status \"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc\": rpc error: code = NotFound desc = could not find container \"2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc\": container with ID starting with 2f8145ea22f3855e20f4efeebcb5ecc877c472fcec2041fddfde7130699765cc not found: ID does not exist"
Sep 07 16:42:13.337340 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.337314    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:42:13.342737 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.342717    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd6c5a-7d9657dcf9-ftxcz"]
Sep 07 16:42:13.889016 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:13.888984    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" path="/var/lib/kubelet/pods/3c7e826c-6d75-4cd0-9a4e-f75652242ac0/volumes"
Sep 07 16:42:18.438626 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:18.438590    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:42:28.892263 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.892223    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:42:28.915131 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:42:28.915612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915594    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15b83ab3-b771-495b-9f3f-b656b564269f" containerName="maas-api"
Sep 07 16:42:28.915784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915614    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15b83ab3-b771-495b-9f3f-b656b564269f" containerName="maas-api"
Sep 07 16:42:28.915784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915627    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" containerName="maas-api"
Sep 07 16:42:28.915784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915636    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" containerName="maas-api"
Sep 07 16:42:28.915784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915644    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c5b27d9-c019-47ed-a155-be1ce72a0154" containerName="maas-api"
Sep 07 16:42:28.915784 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915651    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c5b27d9-c019-47ed-a155-be1ce72a0154" containerName="maas-api"
Sep 07 16:42:28.916041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915791    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c5b27d9-c019-47ed-a155-be1ce72a0154" containerName="maas-api"
Sep 07 16:42:28.916041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915805    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7e826c-6d75-4cd0-9a4e-f75652242ac0" containerName="maas-api"
Sep 07 16:42:28.916041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.915814    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15b83ab3-b771-495b-9f3f-b656b564269f" containerName="maas-api"
Sep 07 16:42:28.920114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.920090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:28.922902 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.922880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-4180e8-serving-cert\""
Sep 07 16:42:28.940087 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:28.940059    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:42:29.004511 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.004482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.004720 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.004524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jzcf\" (UniqueName: \"kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.004720 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.004546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.004720 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.004664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.105140 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.105108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.105140 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.105145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jzcf\" (UniqueName: \"kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.105332 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.105163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.105332 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.105222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.105955 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.105932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.107676 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.107654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.107787 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.107693    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.112163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.112140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jzcf\" (UniqueName: \"kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf\") pod \"maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.230869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.230838    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:29.511277 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.511249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:42:29.513596 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:42:29.513569    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd816f300_8b8e_48db_b135_a2b150d86063.slice/crio-b7c77d633f14cc00aee9b3913f9ebedfc2d7a91ef984c5aaadd46a1717c6773c WatchSource:0}: Error finding container b7c77d633f14cc00aee9b3913f9ebedfc2d7a91ef984c5aaadd46a1717c6773c: Status 404 returned error can't find the container with id b7c77d633f14cc00aee9b3913f9ebedfc2d7a91ef984c5aaadd46a1717c6773c
Sep 07 16:42:29.724676 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.724644    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:42:29.724907 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.724881    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" podUID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" containerName="maas-api" containerID="cri-o://9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b" gracePeriod=30
Sep 07 16:42:29.970709 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:29.970686    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:42:30.116242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.116163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls\") pod \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") "
Sep 07 16:42:30.116242 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.116193    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs\") pod \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") "
Sep 07 16:42:30.116466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.116287    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6xsx\" (UniqueName: \"kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx\") pod \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") "
Sep 07 16:42:30.116466 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.116360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca\") pod \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\" (UID: \"7db88c6a-b94b-43b7-a94a-6be8b37d4241\") "
Sep 07 16:42:30.116797 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.116718    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7db88c6a-b94b-43b7-a94a-6be8b37d4241" (UID: "7db88c6a-b94b-43b7-a94a-6be8b37d4241"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:42:30.118527 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.118502    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7db88c6a-b94b-43b7-a94a-6be8b37d4241" (UID: "7db88c6a-b94b-43b7-a94a-6be8b37d4241"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:42:30.118616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.118526    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7db88c6a-b94b-43b7-a94a-6be8b37d4241" (UID: "7db88c6a-b94b-43b7-a94a-6be8b37d4241"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:42:30.118616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.118566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx" (OuterVolumeSpecName: "kube-api-access-v6xsx") pod "7db88c6a-b94b-43b7-a94a-6be8b37d4241" (UID: "7db88c6a-b94b-43b7-a94a-6be8b37d4241"). InnerVolumeSpecName "kube-api-access-v6xsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:42:30.217988 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.217953    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:30.217988 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.217980    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7db88c6a-b94b-43b7-a94a-6be8b37d4241-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:30.217988 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.217990    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6xsx\" (UniqueName: \"kubernetes.io/projected/7db88c6a-b94b-43b7-a94a-6be8b37d4241-kube-api-access-v6xsx\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:30.218211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.218000    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db88c6a-b94b-43b7-a94a-6be8b37d4241-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:42:30.431848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.431727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" event={"ID":"d816f300-8b8e-48db-b135-a2b150d86063","Type":"ContainerStarted","Data":"d100959e4983a5a6d16eea324c1ff28c0fd5bcbb639672873001750c7c6f7741"}
Sep 07 16:42:30.431848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.431829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:30.432057 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.431852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" event={"ID":"d816f300-8b8e-48db-b135-a2b150d86063","Type":"ContainerStarted","Data":"b7c77d633f14cc00aee9b3913f9ebedfc2d7a91ef984c5aaadd46a1717c6773c"}
Sep 07 16:42:30.433163 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.433138    2579 generic.go:356] "Generic (PLEG): container finished" podID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" containerID="9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b" exitCode=0
Sep 07 16:42:30.433271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.433170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" event={"ID":"7db88c6a-b94b-43b7-a94a-6be8b37d4241","Type":"ContainerDied","Data":"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"}
Sep 07 16:42:30.433271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.433173    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"
Sep 07 16:42:30.433271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.433211    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h" event={"ID":"7db88c6a-b94b-43b7-a94a-6be8b37d4241","Type":"ContainerDied","Data":"8afca7f4d4c28febe97d1427d08c601b820a78a1e6bb9b4959262b80ba9c2f78"}
Sep 07 16:42:30.433271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.433233    2579 scope.go:117] "RemoveContainer" containerID="9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"
Sep 07 16:42:30.442611 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.442595    2579 scope.go:117] "RemoveContainer" containerID="9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"
Sep 07 16:42:30.442918 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:42:30.442892    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b\": container with ID starting with 9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b not found: ID does not exist" containerID="9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"
Sep 07 16:42:30.442982 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.442915    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b"} err="failed to get container status \"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b\": rpc error: code = NotFound desc = could not find container \"9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b\": container with ID starting with 9aeed0b28d7e950072bd9a4d3ff306dd173831cea53f06c0679264d543df2a8b not found: ID does not exist"
Sep 07 16:42:30.446644 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.446604    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" podStartSLOduration=2.027259563 podStartE2EDuration="2.446591375s" totalImagesPullingTime="419.331812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:42:30.446006721 +0000 UTC m=+1389.090302055" watchObservedRunningTime="2026-09-07 16:42:30.446591375 +0000 UTC m=+1389.090886684"
Sep 07 16:42:30.460736 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.460708    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:42:30.464661 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:30.464642    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a9a21-6947f4d649-bkg5h"]
Sep 07 16:42:31.889061 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:31.889030    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" path="/var/lib/kubelet/pods/7db88c6a-b94b-43b7-a94a-6be8b37d4241/volumes"
Sep 07 16:42:36.446326 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:36.446300    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:42:39.430155 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:39.430116    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:42:58.034857 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.034802    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:42:58.035853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.035578    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" containerName="maas-api"
Sep 07 16:42:58.035853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.035599    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" containerName="maas-api"
Sep 07 16:42:58.035853 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.035771    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7db88c6a-b94b-43b7-a94a-6be8b37d4241" containerName="maas-api"
Sep 07 16:42:58.039136 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.039112    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.041188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.041167    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6a45e8-serving-cert\""
Sep 07 16:42:58.049920 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.049897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:42:58.190411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.190375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.190411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.190413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.190700 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.190438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.190700 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.190533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbhq\" (UniqueName: \"kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.291337 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.291243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.291337 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.291287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.291337 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.291319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.291578 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.291353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbhq\" (UniqueName: \"kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.292038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.292008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.293929 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.293901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.294041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.293953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.298611 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.298579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbhq\" (UniqueName: \"kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq\") pod \"maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.350671 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.350632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:58.631778 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:58.631727    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:42:58.634844 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:42:58.634795    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461ab72a_ecc5_495b_88fc_ec4b61675305.slice/crio-f80c5ed474573cf495f2af4952d107b355b62d52c273b68693bf1954076cf7ff WatchSource:0}: Error finding container f80c5ed474573cf495f2af4952d107b355b62d52c273b68693bf1954076cf7ff: Status 404 returned error can't find the container with id f80c5ed474573cf495f2af4952d107b355b62d52c273b68693bf1954076cf7ff
Sep 07 16:42:59.615511 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:59.615467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" event={"ID":"461ab72a-ecc5-495b-88fc-ec4b61675305","Type":"ContainerStarted","Data":"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"}
Sep 07 16:42:59.616024 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:59.615549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" event={"ID":"461ab72a-ecc5-495b-88fc-ec4b61675305","Type":"ContainerStarted","Data":"f80c5ed474573cf495f2af4952d107b355b62d52c273b68693bf1954076cf7ff"}
Sep 07 16:42:59.616024 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:59.615604    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:42:59.632570 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:42:59.632520    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" podStartSLOduration=1.315463595 podStartE2EDuration="1.632506303s" totalImagesPullingTime="317.042708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:42:59.629594443 +0000 UTC m=+1418.273889753" watchObservedRunningTime="2026-09-07 16:42:59.632506303 +0000 UTC m=+1418.276801611"
Sep 07 16:43:03.434026 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:03.433985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:43:05.626598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:05.626567    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:43:22.115019 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:22.114949    2579 scope.go:117] "RemoveContainer" containerID="3a71e92fcfbc79669c37fabe38ed4186f6b3de5f0b52fe3cc4a7a1d3f394f93c"
Sep 07 16:43:22.123914 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:22.123895    2579 scope.go:117] "RemoveContainer" containerID="ffe38a74586675d3e3bc9e736e37bcfc10e4399be4cdab23960c3cc10d5faec3"
Sep 07 16:43:26.692588 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:26.692554    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:43:34.938265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:43:34.938232    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:44:04.029125 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:04.029081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:44:09.762985 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:09.762955    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:44:14.406214 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.406156    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:14.411482 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.411456    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.414146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.414125    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-1cc79d-serving-cert\""
Sep 07 16:44:14.417243 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.417216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:14.497393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.497357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.497549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.497416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.497549 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.497488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kl57\" (UniqueName: \"kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.497627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.497567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.598320 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.598282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.598521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.598343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.598521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.598382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.598521 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.598404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kl57\" (UniqueName: \"kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.599160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.599085    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.600956 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.600926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.601050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.600954    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.605160 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.605138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kl57\" (UniqueName: \"kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57\") pod \"maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:14.725628 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:14.725599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:15.004832 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:15.004803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:15.005932 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:44:15.005896    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6a39eb_571c_46fd_8891_adeb81017bf5.slice/crio-cf0c65326c48404b46cb31640aadc9e469ee6639a04bd2466049a7a7e9f2f18d WatchSource:0}: Error finding container cf0c65326c48404b46cb31640aadc9e469ee6639a04bd2466049a7a7e9f2f18d: Status 404 returned error can't find the container with id cf0c65326c48404b46cb31640aadc9e469ee6639a04bd2466049a7a7e9f2f18d
Sep 07 16:44:15.085496 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:15.085436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" event={"ID":"af6a39eb-571c-46fd-8891-adeb81017bf5","Type":"ContainerStarted","Data":"cf0c65326c48404b46cb31640aadc9e469ee6639a04bd2466049a7a7e9f2f18d"}
Sep 07 16:44:16.092949 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:16.092911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" event={"ID":"af6a39eb-571c-46fd-8891-adeb81017bf5","Type":"ContainerStarted","Data":"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"}
Sep 07 16:44:16.093328 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:16.092979    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:16.109323 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:16.109273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" podStartSLOduration=1.749040929 podStartE2EDuration="2.10925724s" totalImagesPullingTime="360.216311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:44:16.107323468 +0000 UTC m=+1494.751618801" watchObservedRunningTime="2026-09-07 16:44:16.10925724 +0000 UTC m=+1494.753552548"
Sep 07 16:44:21.956144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:21.956118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:44:21.961401 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:21.961375    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:44:22.103616 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:22.103587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:22.240723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:22.240637    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:44:44.874174 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:44.874090    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:44.874741 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:44.874329    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" podUID="af6a39eb-571c-46fd-8891-adeb81017bf5" containerName="maas-api" containerID="cri-o://37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d" gracePeriod=30
Sep 07 16:44:45.137172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.137109    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:45.277406 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277369    2579 generic.go:356] "Generic (PLEG): container finished" podID="af6a39eb-571c-46fd-8891-adeb81017bf5" containerID="37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d" exitCode=0
Sep 07 16:44:45.277602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" event={"ID":"af6a39eb-571c-46fd-8891-adeb81017bf5","Type":"ContainerDied","Data":"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"}
Sep 07 16:44:45.277602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7" event={"ID":"af6a39eb-571c-46fd-8891-adeb81017bf5","Type":"ContainerDied","Data":"cf0c65326c48404b46cb31640aadc9e469ee6639a04bd2466049a7a7e9f2f18d"}
Sep 07 16:44:45.277602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277540    2579 scope.go:117] "RemoveContainer" containerID="37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"
Sep 07 16:44:45.277602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277556    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls\") pod \"af6a39eb-571c-46fd-8891-adeb81017bf5\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") "
Sep 07 16:44:45.277602 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277593    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs\") pod \"af6a39eb-571c-46fd-8891-adeb81017bf5\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") "
Sep 07 16:44:45.277895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277670    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca\") pod \"af6a39eb-571c-46fd-8891-adeb81017bf5\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") "
Sep 07 16:44:45.277895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277683    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"
Sep 07 16:44:45.277895 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.277738    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kl57\" (UniqueName: \"kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57\") pod \"af6a39eb-571c-46fd-8891-adeb81017bf5\" (UID: \"af6a39eb-571c-46fd-8891-adeb81017bf5\") "
Sep 07 16:44:45.278297 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.278236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af6a39eb-571c-46fd-8891-adeb81017bf5" (UID: "af6a39eb-571c-46fd-8891-adeb81017bf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:44:45.280465 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.280420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af6a39eb-571c-46fd-8891-adeb81017bf5" (UID: "af6a39eb-571c-46fd-8891-adeb81017bf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:44:45.281150 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.280839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57" (OuterVolumeSpecName: "kube-api-access-4kl57") pod "af6a39eb-571c-46fd-8891-adeb81017bf5" (UID: "af6a39eb-571c-46fd-8891-adeb81017bf5"). InnerVolumeSpecName "kube-api-access-4kl57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:44:45.281150 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.281114    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "af6a39eb-571c-46fd-8891-adeb81017bf5" (UID: "af6a39eb-571c-46fd-8891-adeb81017bf5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:44:45.300856 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.300830    2579 scope.go:117] "RemoveContainer" containerID="37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"
Sep 07 16:44:45.301158 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:44:45.301139    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d\": container with ID starting with 37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d not found: ID does not exist" containerID="37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"
Sep 07 16:44:45.301224 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.301168    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d"} err="failed to get container status \"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d\": rpc error: code = NotFound desc = could not find container \"37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d\": container with ID starting with 37ccfff868af9f48191eab5c0a492c8dde55a01db0c8e8b4ea646b9d3f1a1f9d not found: ID does not exist"
Sep 07 16:44:45.378964 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.378924    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6a39eb-571c-46fd-8891-adeb81017bf5-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:44:45.378964 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.378961    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kl57\" (UniqueName: \"kubernetes.io/projected/af6a39eb-571c-46fd-8891-adeb81017bf5-kube-api-access-4kl57\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:44:45.379144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.378977    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:44:45.379144 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.378990    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6a39eb-571c-46fd-8891-adeb81017bf5-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:44:45.602622 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.602567    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:45.606516 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.606490    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1cc79d-788fd9c474-mg2f7"]
Sep 07 16:44:45.890585 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:45.890509    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af6a39eb-571c-46fd-8891-adeb81017bf5" path="/var/lib/kubelet/pods/af6a39eb-571c-46fd-8891-adeb81017bf5/volumes"
Sep 07 16:44:52.338621 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:44:52.338586    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:45:10.033633 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.033604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:45:10.243290 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.243258    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:45:10.243603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.243513    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" podUID="d816f300-8b8e-48db-b135-a2b150d86063" containerName="maas-api" containerID="cri-o://d100959e4983a5a6d16eea324c1ff28c0fd5bcbb639672873001750c7c6f7741" gracePeriod=30
Sep 07 16:45:10.429705 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.429669    2579 generic.go:356] "Generic (PLEG): container finished" podID="d816f300-8b8e-48db-b135-a2b150d86063" containerID="d100959e4983a5a6d16eea324c1ff28c0fd5bcbb639672873001750c7c6f7741" exitCode=0
Sep 07 16:45:10.429866 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.429773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" event={"ID":"d816f300-8b8e-48db-b135-a2b150d86063","Type":"ContainerDied","Data":"d100959e4983a5a6d16eea324c1ff28c0fd5bcbb639672873001750c7c6f7741"}
Sep 07 16:45:10.496367 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.496342    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:45:10.602541 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.602512    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls\") pod \"d816f300-8b8e-48db-b135-a2b150d86063\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") "
Sep 07 16:45:10.602706 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.602571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jzcf\" (UniqueName: \"kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf\") pod \"d816f300-8b8e-48db-b135-a2b150d86063\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") "
Sep 07 16:45:10.602706 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.602599    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca\") pod \"d816f300-8b8e-48db-b135-a2b150d86063\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") "
Sep 07 16:45:10.602706 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.602635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs\") pod \"d816f300-8b8e-48db-b135-a2b150d86063\" (UID: \"d816f300-8b8e-48db-b135-a2b150d86063\") "
Sep 07 16:45:10.603079 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.603048    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d816f300-8b8e-48db-b135-a2b150d86063" (UID: "d816f300-8b8e-48db-b135-a2b150d86063"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:45:10.604951 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.604921    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d816f300-8b8e-48db-b135-a2b150d86063" (UID: "d816f300-8b8e-48db-b135-a2b150d86063"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:45:10.605055 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.604961    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d816f300-8b8e-48db-b135-a2b150d86063" (UID: "d816f300-8b8e-48db-b135-a2b150d86063"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:45:10.605055 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.604986    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf" (OuterVolumeSpecName: "kube-api-access-5jzcf") pod "d816f300-8b8e-48db-b135-a2b150d86063" (UID: "d816f300-8b8e-48db-b135-a2b150d86063"). InnerVolumeSpecName "kube-api-access-5jzcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:45:10.703613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.703564    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jzcf\" (UniqueName: \"kubernetes.io/projected/d816f300-8b8e-48db-b135-a2b150d86063-kube-api-access-5jzcf\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:10.703613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.703609    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d816f300-8b8e-48db-b135-a2b150d86063-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:10.703613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.703618    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:10.703828 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:10.703627    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d816f300-8b8e-48db-b135-a2b150d86063-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:11.437411 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.437380    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"
Sep 07 16:45:11.437862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.437415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58" event={"ID":"d816f300-8b8e-48db-b135-a2b150d86063","Type":"ContainerDied","Data":"b7c77d633f14cc00aee9b3913f9ebedfc2d7a91ef984c5aaadd46a1717c6773c"}
Sep 07 16:45:11.437862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.437505    2579 scope.go:117] "RemoveContainer" containerID="d100959e4983a5a6d16eea324c1ff28c0fd5bcbb639672873001750c7c6f7741"
Sep 07 16:45:11.457551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.457518    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:45:11.461540 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.461517    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4180e8-7fb477b6b6-9sc58"]
Sep 07 16:45:11.889606 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:11.889568    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d816f300-8b8e-48db-b135-a2b150d86063" path="/var/lib/kubelet/pods/d816f300-8b8e-48db-b135-a2b150d86063/volumes"
Sep 07 16:45:19.327931 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:19.327895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:45:57.046511 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.046474    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:45:57.047115 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.046779    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" podUID="461ab72a-ecc5-495b-88fc-ec4b61675305" containerName="maas-api" containerID="cri-o://67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2" gracePeriod=30
Sep 07 16:45:57.308612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.308541    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:45:57.438241 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.438209    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs\") pod \"461ab72a-ecc5-495b-88fc-ec4b61675305\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") "
Sep 07 16:45:57.438415 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.438318    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca\") pod \"461ab72a-ecc5-495b-88fc-ec4b61675305\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") "
Sep 07 16:45:57.438415 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.438346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmbhq\" (UniqueName: \"kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq\") pod \"461ab72a-ecc5-495b-88fc-ec4b61675305\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") "
Sep 07 16:45:57.438415 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.438387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls\") pod \"461ab72a-ecc5-495b-88fc-ec4b61675305\" (UID: \"461ab72a-ecc5-495b-88fc-ec4b61675305\") "
Sep 07 16:45:57.438670 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.438645    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "461ab72a-ecc5-495b-88fc-ec4b61675305" (UID: "461ab72a-ecc5-495b-88fc-ec4b61675305"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:45:57.440544 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.440517    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "461ab72a-ecc5-495b-88fc-ec4b61675305" (UID: "461ab72a-ecc5-495b-88fc-ec4b61675305"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:45:57.440658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.440542    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq" (OuterVolumeSpecName: "kube-api-access-vmbhq") pod "461ab72a-ecc5-495b-88fc-ec4b61675305" (UID: "461ab72a-ecc5-495b-88fc-ec4b61675305"). InnerVolumeSpecName "kube-api-access-vmbhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:45:57.440658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.440574    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "461ab72a-ecc5-495b-88fc-ec4b61675305" (UID: "461ab72a-ecc5-495b-88fc-ec4b61675305"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:45:57.539788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.539724    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/461ab72a-ecc5-495b-88fc-ec4b61675305-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:57.539788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.539783    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmbhq\" (UniqueName: \"kubernetes.io/projected/461ab72a-ecc5-495b-88fc-ec4b61675305-kube-api-access-vmbhq\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:57.539788 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.539794    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:57.540033 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.539803    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/461ab72a-ecc5-495b-88fc-ec4b61675305-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:45:57.711099 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.711063    2579 generic.go:356] "Generic (PLEG): container finished" podID="461ab72a-ecc5-495b-88fc-ec4b61675305" containerID="67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2" exitCode=0
Sep 07 16:45:57.711265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.711113    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"
Sep 07 16:45:57.711265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.711143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" event={"ID":"461ab72a-ecc5-495b-88fc-ec4b61675305","Type":"ContainerDied","Data":"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"}
Sep 07 16:45:57.711265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.711198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5" event={"ID":"461ab72a-ecc5-495b-88fc-ec4b61675305","Type":"ContainerDied","Data":"f80c5ed474573cf495f2af4952d107b355b62d52c273b68693bf1954076cf7ff"}
Sep 07 16:45:57.711265 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.711221    2579 scope.go:117] "RemoveContainer" containerID="67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"
Sep 07 16:45:57.721391 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.721373    2579 scope.go:117] "RemoveContainer" containerID="67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"
Sep 07 16:45:57.721689 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:45:57.721662    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2\": container with ID starting with 67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2 not found: ID does not exist" containerID="67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"
Sep 07 16:45:57.721830 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.721691    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2"} err="failed to get container status \"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2\": rpc error: code = NotFound desc = could not find container \"67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2\": container with ID starting with 67698617f03b6de656a0170530ae0424a46bf2f879df9dd50c4e429aa2cd0bb2 not found: ID does not exist"
Sep 07 16:45:57.731508 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.731484    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:45:57.737356 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.737334    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6a45e8-7699bb44bd-cxlm5"]
Sep 07 16:45:57.889584 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:45:57.889544    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="461ab72a-ecc5-495b-88fc-ec4b61675305" path="/var/lib/kubelet/pods/461ab72a-ecc5-495b-88fc-ec4b61675305/volumes"
Sep 07 16:46:01.333090 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:01.333060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:46:14.674816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:14.674709    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:46:14.675258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:14.674987    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" podUID="eca529b6-de3d-4085-9d12-43b6de5a11fc" containerName="maas-api" containerID="cri-o://9d5d2afaf6415e44021ce8bcca37f84a47b7ac7c649a7028462cd695a8446b64" gracePeriod=30
Sep 07 16:46:14.817331 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:14.817294    2579 generic.go:356] "Generic (PLEG): container finished" podID="eca529b6-de3d-4085-9d12-43b6de5a11fc" containerID="9d5d2afaf6415e44021ce8bcca37f84a47b7ac7c649a7028462cd695a8446b64" exitCode=0
Sep 07 16:46:14.817532 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:14.817383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" event={"ID":"eca529b6-de3d-4085-9d12-43b6de5a11fc","Type":"ContainerDied","Data":"9d5d2afaf6415e44021ce8bcca37f84a47b7ac7c649a7028462cd695a8446b64"}
Sep 07 16:46:14.941501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:14.941432    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:46:15.105070 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.105037    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7d57\" (UniqueName: \"kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57\") pod \"eca529b6-de3d-4085-9d12-43b6de5a11fc\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") "
Sep 07 16:46:15.105070 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.105074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls\") pod \"eca529b6-de3d-4085-9d12-43b6de5a11fc\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") "
Sep 07 16:46:15.105342 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.105115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca\") pod \"eca529b6-de3d-4085-9d12-43b6de5a11fc\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") "
Sep 07 16:46:15.105342 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.105219    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs\") pod \"eca529b6-de3d-4085-9d12-43b6de5a11fc\" (UID: \"eca529b6-de3d-4085-9d12-43b6de5a11fc\") "
Sep 07 16:46:15.105583 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.105546    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eca529b6-de3d-4085-9d12-43b6de5a11fc" (UID: "eca529b6-de3d-4085-9d12-43b6de5a11fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:46:15.107414 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.107390    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eca529b6-de3d-4085-9d12-43b6de5a11fc" (UID: "eca529b6-de3d-4085-9d12-43b6de5a11fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:15.107501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.107443    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57" (OuterVolumeSpecName: "kube-api-access-p7d57") pod "eca529b6-de3d-4085-9d12-43b6de5a11fc" (UID: "eca529b6-de3d-4085-9d12-43b6de5a11fc"). InnerVolumeSpecName "kube-api-access-p7d57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:46:15.107501 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.107480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eca529b6-de3d-4085-9d12-43b6de5a11fc" (UID: "eca529b6-de3d-4085-9d12-43b6de5a11fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:15.206246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.206208    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:15.206246 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.206242    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7d57\" (UniqueName: \"kubernetes.io/projected/eca529b6-de3d-4085-9d12-43b6de5a11fc-kube-api-access-p7d57\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:15.206458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.206257    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eca529b6-de3d-4085-9d12-43b6de5a11fc-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:15.206458 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.206269    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eca529b6-de3d-4085-9d12-43b6de5a11fc-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:15.824228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.824195    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"
Sep 07 16:46:15.824698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.824231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj" event={"ID":"eca529b6-de3d-4085-9d12-43b6de5a11fc","Type":"ContainerDied","Data":"3f68711977ac086fdac588d86564a2d2039b6b26278752769ea041db11a5035b"}
Sep 07 16:46:15.824698 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.824298    2579 scope.go:117] "RemoveContainer" containerID="9d5d2afaf6415e44021ce8bcca37f84a47b7ac7c649a7028462cd695a8446b64"
Sep 07 16:46:15.846594 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.846564    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:46:15.852848 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.852823    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8d59e2-6ff94bcf4b-m7pnj"]
Sep 07 16:46:15.889596 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:15.889564    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca529b6-de3d-4085-9d12-43b6de5a11fc" path="/var/lib/kubelet/pods/eca529b6-de3d-4085-9d12-43b6de5a11fc/volumes"
Sep 07 16:46:18.837785 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:18.837712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:46:32.260292 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.260258    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:46:32.260957 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.260599    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" podUID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" containerName="maas-api" containerID="cri-o://e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95" gracePeriod=30
Sep 07 16:46:32.516226 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.516167    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:46:32.551202 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.551173    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls\") pod \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") "
Sep 07 16:46:32.551379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.551262    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs\") pod \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") "
Sep 07 16:46:32.551379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.551291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca\") pod \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") "
Sep 07 16:46:32.551379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.551333    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6gfl\" (UniqueName: \"kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl\") pod \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\" (UID: \"d3592795-3ac0-4d16-82e7-37a8cfb5cd81\") "
Sep 07 16:46:32.551716 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.551688    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3592795-3ac0-4d16-82e7-37a8cfb5cd81" (UID: "d3592795-3ac0-4d16-82e7-37a8cfb5cd81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:46:32.553528 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.553499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3592795-3ac0-4d16-82e7-37a8cfb5cd81" (UID: "d3592795-3ac0-4d16-82e7-37a8cfb5cd81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:32.553658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.553549    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d3592795-3ac0-4d16-82e7-37a8cfb5cd81" (UID: "d3592795-3ac0-4d16-82e7-37a8cfb5cd81"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:32.553864 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.553839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl" (OuterVolumeSpecName: "kube-api-access-l6gfl") pod "d3592795-3ac0-4d16-82e7-37a8cfb5cd81" (UID: "d3592795-3ac0-4d16-82e7-37a8cfb5cd81"). InnerVolumeSpecName "kube-api-access-l6gfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:46:32.652530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.652488    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:32.652530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.652519    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:32.652530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.652530    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6gfl\" (UniqueName: \"kubernetes.io/projected/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-kube-api-access-l6gfl\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:32.652862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.652539    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3592795-3ac0-4d16-82e7-37a8cfb5cd81-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:32.927703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.927601    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" containerID="e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95" exitCode=0
Sep 07 16:46:32.927703 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.927683    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"
Sep 07 16:46:32.927921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.927699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" event={"ID":"d3592795-3ac0-4d16-82e7-37a8cfb5cd81","Type":"ContainerDied","Data":"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"}
Sep 07 16:46:32.927921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.927773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d" event={"ID":"d3592795-3ac0-4d16-82e7-37a8cfb5cd81","Type":"ContainerDied","Data":"72e384794b6f46c9c323b11f9ee5bceb6a39ee81bf4e56a7dbcd243ec5324bc4"}
Sep 07 16:46:32.927921 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.927800    2579 scope.go:117] "RemoveContainer" containerID="e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"
Sep 07 16:46:32.937639 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.937624    2579 scope.go:117] "RemoveContainer" containerID="e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"
Sep 07 16:46:32.937942 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:46:32.937921    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95\": container with ID starting with e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95 not found: ID does not exist" containerID="e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"
Sep 07 16:46:32.938025 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.937948    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95"} err="failed to get container status \"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95\": rpc error: code = NotFound desc = could not find container \"e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95\": container with ID starting with e859701fbc26134e23605bb5ad25763ec7482f502e9c6e96f75bf75e96ca7c95 not found: ID does not exist"
Sep 07 16:46:32.949468 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.949439    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:46:32.953202 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:32.953182    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7aca0f-7bd9c59488-9s45d"]
Sep 07 16:46:33.889467 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:33.889435    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" path="/var/lib/kubelet/pods/d3592795-3ac0-4d16-82e7-37a8cfb5cd81/volumes"
Sep 07 16:46:35.441052 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:35.441013    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:46:41.334421 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.334391    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:46:41.334901 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.334640    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" podUID="d6a0b68a-29be-4542-96a4-32b4d46ab684" containerName="manager" containerID="cri-o://70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07" gracePeriod=30
Sep 07 16:46:41.582336 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.582304    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:46:41.638366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.638289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs\") pod \"d6a0b68a-29be-4542-96a4-32b4d46ab684\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") "
Sep 07 16:46:41.638366 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.638365    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert\") pod \"d6a0b68a-29be-4542-96a4-32b4d46ab684\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") "
Sep 07 16:46:41.638557 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.638533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbwg2\" (UniqueName: \"kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2\") pod \"d6a0b68a-29be-4542-96a4-32b4d46ab684\" (UID: \"d6a0b68a-29be-4542-96a4-32b4d46ab684\") "
Sep 07 16:46:41.640649 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.640622    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2" (OuterVolumeSpecName: "kube-api-access-mbwg2") pod "d6a0b68a-29be-4542-96a4-32b4d46ab684" (UID: "d6a0b68a-29be-4542-96a4-32b4d46ab684"). InnerVolumeSpecName "kube-api-access-mbwg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:46:41.640878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.640855    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d6a0b68a-29be-4542-96a4-32b4d46ab684" (UID: "d6a0b68a-29be-4542-96a4-32b4d46ab684"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:41.640959 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.640881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6a0b68a-29be-4542-96a4-32b4d46ab684" (UID: "d6a0b68a-29be-4542-96a4-32b4d46ab684"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:46:41.739890 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.739860    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbwg2\" (UniqueName: \"kubernetes.io/projected/d6a0b68a-29be-4542-96a4-32b4d46ab684-kube-api-access-mbwg2\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:41.739890 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.739886    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:41.739890 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.739895    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6a0b68a-29be-4542-96a4-32b4d46ab684-webhook-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:46:41.983773 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.983722    2579 generic.go:356] "Generic (PLEG): container finished" podID="d6a0b68a-29be-4542-96a4-32b4d46ab684" containerID="70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07" exitCode=0
Sep 07 16:46:41.983968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.983792    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm"
Sep 07 16:46:41.983968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.983806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" event={"ID":"d6a0b68a-29be-4542-96a4-32b4d46ab684","Type":"ContainerDied","Data":"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"}
Sep 07 16:46:41.983968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.983859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-w7nfm" event={"ID":"d6a0b68a-29be-4542-96a4-32b4d46ab684","Type":"ContainerDied","Data":"9c062c191ccdda621155e2c4fc04df94f1ad6a1e9e91d2faed7fd969c0bbe683"}
Sep 07 16:46:41.983968 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.983886    2579 scope.go:117] "RemoveContainer" containerID="70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"
Sep 07 16:46:41.994559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.994538    2579 scope.go:117] "RemoveContainer" containerID="70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"
Sep 07 16:46:41.994893 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:46:41.994865    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07\": container with ID starting with 70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07 not found: ID does not exist" containerID="70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"
Sep 07 16:46:41.994966 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:41.994897    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07"} err="failed to get container status \"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07\": rpc error: code = NotFound desc = could not find container \"70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07\": container with ID starting with 70b68e29e4d3ac1442ef36473ddbd97598cf78d0d858b6eb515940a362a25b07 not found: ID does not exist"
Sep 07 16:46:42.000040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:42.000017    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:46:42.005883 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:42.005859    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-w7nfm"]
Sep 07 16:46:43.141044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:43.141009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:46:43.888693 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:46:43.888659    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6a0b68a-29be-4542-96a4-32b4d46ab684" path="/var/lib/kubelet/pods/d6a0b68a-29be-4542-96a4-32b4d46ab684/volumes"
Sep 07 16:47:22.200733 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:22.200701    2579 scope.go:117] "RemoveContainer" containerID="66ba4f5d24fbd3fc717b1755c83d2b1804c341209d403c4da5164dbcea4732ab"
Sep 07 16:47:42.771180 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771104    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771543    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" containerName="maas-api"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771557    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" containerName="maas-api"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771565    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d816f300-8b8e-48db-b135-a2b150d86063" containerName="maas-api"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771571    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d816f300-8b8e-48db-b135-a2b150d86063" containerName="maas-api"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771583    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a0b68a-29be-4542-96a4-32b4d46ab684" containerName="manager"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771588    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a0b68a-29be-4542-96a4-32b4d46ab684" containerName="manager"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771598    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca529b6-de3d-4085-9d12-43b6de5a11fc" containerName="maas-api"
Sep 07 16:47:42.771600 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771603    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca529b6-de3d-4085-9d12-43b6de5a11fc" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771609    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461ab72a-ecc5-495b-88fc-ec4b61675305" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="461ab72a-ecc5-495b-88fc-ec4b61675305" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771628    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6a39eb-571c-46fd-8891-adeb81017bf5" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771633    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6a39eb-571c-46fd-8891-adeb81017bf5" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771697    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3592795-3ac0-4d16-82e7-37a8cfb5cd81" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771704    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a0b68a-29be-4542-96a4-32b4d46ab684" containerName="manager"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771711    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6a39eb-571c-46fd-8891-adeb81017bf5" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771717    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca529b6-de3d-4085-9d12-43b6de5a11fc" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771723    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d816f300-8b8e-48db-b135-a2b150d86063" containerName="maas-api"
Sep 07 16:47:42.771869 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.771729    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="461ab72a-ecc5-495b-88fc-ec4b61675305" containerName="maas-api"
Sep 07 16:47:42.776154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.776132    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:42.778368 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.778348    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 16:47:42.779151 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.779136    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 16:47:42.779228 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.779194    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 16:47:42.779273 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.779229    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cvp8t\""
Sep 07 16:47:42.785009 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.784987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:47:42.908271 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.908238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5c6\" (UniqueName: \"kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:42.908457 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.908294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:42.908457 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:42.908372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.009357 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.009304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5c6\" (UniqueName: \"kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.009559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.009395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.009559 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.009430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.012050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.012022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.012050 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.012044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.016966 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.016949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5c6\" (UniqueName: \"kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6\") pod \"maas-controller-56db7f4c4c-4zd7g\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") " pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.088178 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.088013    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:43.361506 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.361477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:47:43.363481 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:47:43.363453    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb3a89f_1eba_4e19_bdfc_24dfb8b4c791.slice/crio-86f8bfd47b9495925544d9c742000b8471f7545c648392c9483aad9168af1dfa WatchSource:0}: Error finding container 86f8bfd47b9495925544d9c742000b8471f7545c648392c9483aad9168af1dfa: Status 404 returned error can't find the container with id 86f8bfd47b9495925544d9c742000b8471f7545c648392c9483aad9168af1dfa
Sep 07 16:47:43.364759 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:43.364729    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:47:44.343018 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:44.342979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" event={"ID":"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791","Type":"ContainerStarted","Data":"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"}
Sep 07 16:47:44.343397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:44.343070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" event={"ID":"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791","Type":"ContainerStarted","Data":"86f8bfd47b9495925544d9c742000b8471f7545c648392c9483aad9168af1dfa"}
Sep 07 16:47:44.343397 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:44.343092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:47:44.360222 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:44.360167    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" podStartSLOduration=1.858449389 podStartE2EDuration="2.360148315s" totalImagesPullingTime="501.698926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:47:44.358696269 +0000 UTC m=+1703.002991596" watchObservedRunningTime="2026-09-07 16:47:44.360148315 +0000 UTC m=+1703.004443627"
Sep 07 16:47:55.352146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:47:55.352108    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:48:04.241578 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.241542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:48:04.626350 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.626256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:48:04.626670 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.626622    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" podUID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" containerName="manager" containerID="cri-o://e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95" gracePeriod=30
Sep 07 16:48:04.875264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.875233    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:48:04.893444 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.893142    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs\") pod \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") "
Sep 07 16:48:04.893444 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.893186    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert\") pod \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") "
Sep 07 16:48:04.893444 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.893235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl5c6\" (UniqueName: \"kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6\") pod \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\" (UID: \"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791\") "
Sep 07 16:48:04.896084 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.896056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" (UID: "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:48:04.896362 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.896343    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" (UID: "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:48:04.896429 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.896384    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6" (OuterVolumeSpecName: "kube-api-access-vl5c6") pod "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" (UID: "efb3a89f-1eba-4e19-bdfc-24dfb8b4c791"). InnerVolumeSpecName "kube-api-access-vl5c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:48:04.994096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.994057    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl5c6\" (UniqueName: \"kubernetes.io/projected/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-kube-api-access-vl5c6\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:48:04.994096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.994088    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:48:04.994096 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:04.994097    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791-webhook-cert\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:48:05.464490 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.464454    2579 generic.go:356] "Generic (PLEG): container finished" podID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" containerID="e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95" exitCode=0
Sep 07 16:48:05.464878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.464501    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g"
Sep 07 16:48:05.464878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.464540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" event={"ID":"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791","Type":"ContainerDied","Data":"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"}
Sep 07 16:48:05.464878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.464597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-4zd7g" event={"ID":"efb3a89f-1eba-4e19-bdfc-24dfb8b4c791","Type":"ContainerDied","Data":"86f8bfd47b9495925544d9c742000b8471f7545c648392c9483aad9168af1dfa"}
Sep 07 16:48:05.464878 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.464627    2579 scope.go:117] "RemoveContainer" containerID="e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"
Sep 07 16:48:05.474613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.474595    2579 scope.go:117] "RemoveContainer" containerID="e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"
Sep 07 16:48:05.474922 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:48:05.474901    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95\": container with ID starting with e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95 not found: ID does not exist" containerID="e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"
Sep 07 16:48:05.474994 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.474931    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95"} err="failed to get container status \"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95\": rpc error: code = NotFound desc = could not find container \"e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95\": container with ID starting with e27a0bd9e8c51681f5a2e089626a3b88454096452e22dc9f6b7d2f15171d2b95 not found: ID does not exist"
Sep 07 16:48:05.485321 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.485292    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:48:05.490973 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.490949    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-4zd7g"]
Sep 07 16:48:05.889041 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:05.888967    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" path="/var/lib/kubelet/pods/efb3a89f-1eba-4e19-bdfc-24dfb8b4c791/volumes"
Sep 07 16:48:06.230660 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.230628    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-xl2cx"]
Sep 07 16:48:06.231101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.231085    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" containerName="manager"
Sep 07 16:48:06.231146 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.231104    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" containerName="manager"
Sep 07 16:48:06.231187 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.231177    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb3a89f-1eba-4e19-bdfc-24dfb8b4c791" containerName="manager"
Sep 07 16:48:06.235654 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.235638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.238662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.238629    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 16:48:06.238662 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.238657    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 16:48:06.238880 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.238665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cvp8t\""
Sep 07 16:48:06.238880 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.238665    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 16:48:06.243389 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.243320    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-xl2cx"]
Sep 07 16:48:06.304980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.304946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-metrics-certs\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.304980 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.304982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-webhook-cert\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.305213 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.305010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvc6f\" (UniqueName: \"kubernetes.io/projected/ff017eed-eb10-408f-bc80-3b4c2693d2e7-kube-api-access-hvc6f\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.406095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.406053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-metrics-certs\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.406095 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.406097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-webhook-cert\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.406354 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.406139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvc6f\" (UniqueName: \"kubernetes.io/projected/ff017eed-eb10-408f-bc80-3b4c2693d2e7-kube-api-access-hvc6f\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.408696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.408661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-metrics-certs\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.408696 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.408689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff017eed-eb10-408f-bc80-3b4c2693d2e7-webhook-cert\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.413989 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.413967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvc6f\" (UniqueName: \"kubernetes.io/projected/ff017eed-eb10-408f-bc80-3b4c2693d2e7-kube-api-access-hvc6f\") pod \"maas-controller-56db7f4c4c-xl2cx\" (UID: \"ff017eed-eb10-408f-bc80-3b4c2693d2e7\") " pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.547264 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.547179    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:06.827656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:06.827624    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56db7f4c4c-xl2cx"]
Sep 07 16:48:06.829060 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:48:06.829035    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff017eed_eb10_408f_bc80_3b4c2693d2e7.slice/crio-116a4f9370da5ff51fcbe13654848626b3d490d78761862aab9a0e54472a4529 WatchSource:0}: Error finding container 116a4f9370da5ff51fcbe13654848626b3d490d78761862aab9a0e54472a4529: Status 404 returned error can't find the container with id 116a4f9370da5ff51fcbe13654848626b3d490d78761862aab9a0e54472a4529
Sep 07 16:48:07.479900 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:07.479861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx" event={"ID":"ff017eed-eb10-408f-bc80-3b4c2693d2e7","Type":"ContainerStarted","Data":"8800cf28db38c1a9bbe96ebf4427ae541a312c75ee339b99e4d95bbcafb75ae9"}
Sep 07 16:48:07.480071 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:07.479910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx" event={"ID":"ff017eed-eb10-408f-bc80-3b4c2693d2e7","Type":"ContainerStarted","Data":"116a4f9370da5ff51fcbe13654848626b3d490d78761862aab9a0e54472a4529"}
Sep 07 16:48:07.480071 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:07.479960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:07.499844 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:07.499724    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx" podStartSLOduration=1.086040163 podStartE2EDuration="1.499708218s" totalImagesPullingTime="413.668055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:48:07.497021828 +0000 UTC m=+1726.141317155" watchObservedRunningTime="2026-09-07 16:48:07.499708218 +0000 UTC m=+1726.144003576"
Sep 07 16:48:18.490115 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:18.490083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56db7f4c4c-xl2cx"
Sep 07 16:48:30.234574 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:30.234534    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:48:38.342598 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:38.342560    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:48:45.339958 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:45.339922    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:48:52.535128 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:48:52.535091    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:49:00.051500 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:00.051457    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:49:07.635843 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:07.635808    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:49:18.833425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:18.833341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:49:21.994682 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:21.994655    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:49:22.002612 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:22.002590    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:49:24.432006 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:24.431973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:49:25.902603 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:25.902562    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:49:25.903016 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:25.902846    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" podUID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" containerName="manager" containerID="cri-o://b5920366b24f3cd0d97599d9c92209340fe780a5e7368acb66bdfa5fa5c2da36" gracePeriod=10
Sep 07 16:49:27.949829 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:27.949796    2579 generic.go:356] "Generic (PLEG): container finished" podID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" containerID="b5920366b24f3cd0d97599d9c92209340fe780a5e7368acb66bdfa5fa5c2da36" exitCode=0
Sep 07 16:49:27.950114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:27.949841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" event={"ID":"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d","Type":"ContainerDied","Data":"b5920366b24f3cd0d97599d9c92209340fe780a5e7368acb66bdfa5fa5c2da36"}
Sep 07 16:49:27.950114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:27.949889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g" event={"ID":"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d","Type":"ContainerDied","Data":"420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5"}
Sep 07 16:49:27.950114 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:27.949912    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="420a959f944ddc7fb7c10640766aa7dcb1bace6a583e1caf5e8b0ab87b051ba5"
Sep 07 16:49:27.960379 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:27.960357    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:49:28.052816 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.052721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume\") pod \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") "
Sep 07 16:49:28.052957 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.052844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mfj5\" (UniqueName: \"kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5\") pod \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\" (UID: \"af76e0d4-3f97-4f8c-95aa-d48f24d0e60d\") "
Sep 07 16:49:28.053101 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.053079    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" (UID: "af76e0d4-3f97-4f8c-95aa-d48f24d0e60d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 16:49:28.054916 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.054887    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5" (OuterVolumeSpecName: "kube-api-access-6mfj5") pod "af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" (UID: "af76e0d4-3f97-4f8c-95aa-d48f24d0e60d"). InnerVolumeSpecName "kube-api-access-6mfj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:49:28.154460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.154417    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mfj5\" (UniqueName: \"kubernetes.io/projected/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-kube-api-access-6mfj5\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:49:28.154460 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.154455    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d-extensions-socket-volume\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:49:28.954806 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.954769    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"
Sep 07 16:49:28.976312 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.976280    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:49:28.980304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:28.980278    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-cd64g"]
Sep 07 16:49:29.887590 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:49:29.887556    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" path="/var/lib/kubelet/pods/af76e0d4-3f97-4f8c-95aa-d48f24d0e60d/volumes"
Sep 07 16:50:22.253512 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:22.253480    2579 scope.go:117] "RemoveContainer" containerID="b5920366b24f3cd0d97599d9c92209340fe780a5e7368acb66bdfa5fa5c2da36"
Sep 07 16:50:26.631777 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.631707    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"]
Sep 07 16:50:26.632141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.632127    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" containerName="manager"
Sep 07 16:50:26.632188 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.632143    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" containerName="manager"
Sep 07 16:50:26.632230 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.632222    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="af76e0d4-3f97-4f8c-95aa-d48f24d0e60d" containerName="manager"
Sep 07 16:50:26.635284 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.635268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.637805 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.637784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z2dkf\""
Sep 07 16:50:26.660669 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.660643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"]
Sep 07 16:50:26.675812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.675783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/265862c6-e917-439b-a36d-1882e1855bd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.675962 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.675942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbjt\" (UniqueName: \"kubernetes.io/projected/265862c6-e917-439b-a36d-1882e1855bd6-kube-api-access-nsbjt\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.776563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.776529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/265862c6-e917-439b-a36d-1882e1855bd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.776740 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.776637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbjt\" (UniqueName: \"kubernetes.io/projected/265862c6-e917-439b-a36d-1882e1855bd6-kube-api-access-nsbjt\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.777038 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.777017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/265862c6-e917-439b-a36d-1882e1855bd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.787761 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.787728    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbjt\" (UniqueName: \"kubernetes.io/projected/265862c6-e917-439b-a36d-1882e1855bd6-kube-api-access-nsbjt\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-m2w5l\" (UID: \"265862c6-e917-439b-a36d-1882e1855bd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:26.945097 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:26.945011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:27.235608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:27.235581    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"]
Sep 07 16:50:27.236838 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:50:27.236812    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265862c6_e917_439b_a36d_1882e1855bd6.slice/crio-7df3cdc4cc20487e20bccca6a7050920a6170714cef2ee71b53e158d299d1cb3 WatchSource:0}: Error finding container 7df3cdc4cc20487e20bccca6a7050920a6170714cef2ee71b53e158d299d1cb3: Status 404 returned error can't find the container with id 7df3cdc4cc20487e20bccca6a7050920a6170714cef2ee71b53e158d299d1cb3
Sep 07 16:50:27.285551 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:27.285519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l" event={"ID":"265862c6-e917-439b-a36d-1882e1855bd6","Type":"ContainerStarted","Data":"7df3cdc4cc20487e20bccca6a7050920a6170714cef2ee71b53e158d299d1cb3"}
Sep 07 16:50:28.293580 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:28.293542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l" event={"ID":"265862c6-e917-439b-a36d-1882e1855bd6","Type":"ContainerStarted","Data":"fae232e5a12ef1ec264dfe8aed51e8befa8cc19a3ffab03a69b9b006a0f71ada"}
Sep 07 16:50:28.294135 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:28.293605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:28.338706 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:28.338644    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l" podStartSLOduration=2.338628502 podStartE2EDuration="2.338628502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:50:28.336129893 +0000 UTC m=+1866.980425202" watchObservedRunningTime="2026-09-07 16:50:28.338628502 +0000 UTC m=+1866.982923810"
Sep 07 16:50:39.302689 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:39.302652    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-m2w5l"
Sep 07 16:50:51.567335 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:50:51.567257    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:51:00.136800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:51:00.136730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:51:09.049427 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:51:09.049383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:51:19.151627 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:51:19.151587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:51:45.235205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:51:45.235167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:51:53.742827 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:51:53.742792    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:03.746871 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.746836    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"]
Sep 07 16:52:03.750524 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.750506    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:03.769069 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.769042    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"]
Sep 07 16:52:03.914850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.914816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136fc2b9-f124-4542-9dd7-2ef278af12fb-openshift-service-ca\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:03.914850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.914857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-maas-api-tls\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:03.915119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.914874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-metrics-certs\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:03.915119 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:03.914946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzglj\" (UniqueName: \"kubernetes.io/projected/136fc2b9-f124-4542-9dd7-2ef278af12fb-kube-api-access-xzglj\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.016279 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.016171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136fc2b9-f124-4542-9dd7-2ef278af12fb-openshift-service-ca\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.016279 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.016234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-maas-api-tls\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.016517 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.016413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-metrics-certs\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.016517 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.016448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzglj\" (UniqueName: \"kubernetes.io/projected/136fc2b9-f124-4542-9dd7-2ef278af12fb-kube-api-access-xzglj\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.017172 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.017146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136fc2b9-f124-4542-9dd7-2ef278af12fb-openshift-service-ca\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.019581 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.019515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-metrics-certs\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.019688 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.019584    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136fc2b9-f124-4542-9dd7-2ef278af12fb-maas-api-tls\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.024341 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.024302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzglj\" (UniqueName: \"kubernetes.io/projected/136fc2b9-f124-4542-9dd7-2ef278af12fb-kube-api-access-xzglj\") pod \"maas-api-5946dbd7c4-qc2z2\" (UID: \"136fc2b9-f124-4542-9dd7-2ef278af12fb\") " pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.061796 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.061764    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:04.358205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.358176    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"]
Sep 07 16:52:04.360019 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:52:04.359979    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136fc2b9_f124_4542_9dd7_2ef278af12fb.slice/crio-f724136b387b87c83fec40c4c1d91c7ffab768d77821f4c50e7fab3452b3a679 WatchSource:0}: Error finding container f724136b387b87c83fec40c4c1d91c7ffab768d77821f4c50e7fab3452b3a679: Status 404 returned error can't find the container with id f724136b387b87c83fec40c4c1d91c7ffab768d77821f4c50e7fab3452b3a679
Sep 07 16:52:04.734735 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.734702    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:04.858141 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:04.858100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2" event={"ID":"136fc2b9-f124-4542-9dd7-2ef278af12fb","Type":"ContainerStarted","Data":"f724136b387b87c83fec40c4c1d91c7ffab768d77821f4c50e7fab3452b3a679"}
Sep 07 16:52:05.865250 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:05.865217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2" event={"ID":"136fc2b9-f124-4542-9dd7-2ef278af12fb","Type":"ContainerStarted","Data":"73a7b0c3f562331fa124b7bb8a993bfc8dd0bb734fe1d18d88b35036b4ab601e"}
Sep 07 16:52:05.865619 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:05.865316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:05.911195 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:05.911143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2" podStartSLOduration=2.335330522 podStartE2EDuration="2.91112617s" totalImagesPullingTime="575.795648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:52:05.909940389 +0000 UTC m=+1964.554235697" watchObservedRunningTime="2026-09-07 16:52:05.91112617 +0000 UTC m=+1964.555421478"
Sep 07 16:52:11.344555 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:11.344519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:11.876729 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:11.876699    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5946dbd7c4-qc2z2"
Sep 07 16:52:19.963800 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:19.963690    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:52:19.964314 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:19.963955    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" podUID="6726ca22-7c51-4be7-81e3-23d3b6821a55" containerName="maas-api" containerID="cri-o://5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f" gracePeriod=30
Sep 07 16:52:21.213236 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.213210    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:52:21.275896 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.275865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4h96\" (UniqueName: \"kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96\") pod \"6726ca22-7c51-4be7-81e3-23d3b6821a55\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") "
Sep 07 16:52:21.276055 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.275903    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca\") pod \"6726ca22-7c51-4be7-81e3-23d3b6821a55\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") "
Sep 07 16:52:21.276055 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.275970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls\") pod \"6726ca22-7c51-4be7-81e3-23d3b6821a55\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") "
Sep 07 16:52:21.276055 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.275998    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs\") pod \"6726ca22-7c51-4be7-81e3-23d3b6821a55\" (UID: \"6726ca22-7c51-4be7-81e3-23d3b6821a55\") "
Sep 07 16:52:21.276340 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.276311    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6726ca22-7c51-4be7-81e3-23d3b6821a55" (UID: "6726ca22-7c51-4be7-81e3-23d3b6821a55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 16:52:21.278157 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.278130    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6726ca22-7c51-4be7-81e3-23d3b6821a55" (UID: "6726ca22-7c51-4be7-81e3-23d3b6821a55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:52:21.278420 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.278396    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96" (OuterVolumeSpecName: "kube-api-access-x4h96") pod "6726ca22-7c51-4be7-81e3-23d3b6821a55" (UID: "6726ca22-7c51-4be7-81e3-23d3b6821a55"). InnerVolumeSpecName "kube-api-access-x4h96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 16:52:21.278617 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.278597    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6726ca22-7c51-4be7-81e3-23d3b6821a55" (UID: "6726ca22-7c51-4be7-81e3-23d3b6821a55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 16:52:21.345347 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.345311    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:21.377072 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.377042    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4h96\" (UniqueName: \"kubernetes.io/projected/6726ca22-7c51-4be7-81e3-23d3b6821a55-kube-api-access-x4h96\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:52:21.377209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.377180    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6726ca22-7c51-4be7-81e3-23d3b6821a55-openshift-service-ca\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:52:21.377209 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.377200    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-maas-api-tls\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:52:21.377338 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.377216    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6726ca22-7c51-4be7-81e3-23d3b6821a55-metrics-certs\") on node \"ip-10-0-135-154.ec2.internal\" DevicePath \"\""
Sep 07 16:52:21.960936 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.960898    2579 generic.go:356] "Generic (PLEG): container finished" podID="6726ca22-7c51-4be7-81e3-23d3b6821a55" containerID="5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f" exitCode=0
Sep 07 16:52:21.961109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.960954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" event={"ID":"6726ca22-7c51-4be7-81e3-23d3b6821a55","Type":"ContainerDied","Data":"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"}
Sep 07 16:52:21.961109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.960969    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"
Sep 07 16:52:21.961109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.961012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc" event={"ID":"6726ca22-7c51-4be7-81e3-23d3b6821a55","Type":"ContainerDied","Data":"d3d7e88678e1514e36df0da4c6d432e75255bca1cb99fd9c0fa32c3b58e89b44"}
Sep 07 16:52:21.961109 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.961035    2579 scope.go:117] "RemoveContainer" containerID="5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"
Sep 07 16:52:21.970862 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.970829    2579 scope.go:117] "RemoveContainer" containerID="5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"
Sep 07 16:52:21.971183 ip-10-0-135-154 kubenswrapper[2579]: E0907 16:52:21.971161    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f\": container with ID starting with 5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f not found: ID does not exist" containerID="5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"
Sep 07 16:52:21.971253 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.971194    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f"} err="failed to get container status \"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f\": rpc error: code = NotFound desc = could not find container \"5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f\": container with ID starting with 5e7f260b000396e5de57f560b7178fe159b2fee73327e27884d7a3a3af29cd0f not found: ID does not exist"
Sep 07 16:52:21.977295 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.977270    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:52:21.982825 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:21.982801    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5868b9dbc4-vlkhc"]
Sep 07 16:52:23.889393 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:23.889355    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6726ca22-7c51-4be7-81e3-23d3b6821a55" path="/var/lib/kubelet/pods/6726ca22-7c51-4be7-81e3-23d3b6821a55/volumes"
Sep 07 16:52:27.629875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:27.629838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:35.126287 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:35.126249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ffvz2"]
Sep 07 16:52:43.562578 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:43.562544    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cs4vd_2ba65f54-d03d-45ab-8a2b-64e68250d677/manager/0.log"
Sep 07 16:52:43.668218 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:43.668185    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5qnzr_9d2e8250-239b-415a-ace9-31be31c0a7b9/manager/0.log"
Sep 07 16:52:43.779597 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:43.779561    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56db7f4c4c-xl2cx_ff017eed-eb10-408f-bc80-3b4c2693d2e7/manager/0.log"
Sep 07 16:52:43.882996 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:43.882882    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n9xgx_e77947cc-2180-414c-b67c-f5e49a013789/server/0.log"
Sep 07 16:52:43.993724 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:43.993686    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cwjtb_7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9/manager/0.log"
Sep 07 16:52:44.105056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:44.105028    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7766cc5489-77jsf_dbee4a3c-3a98-4dfb-a599-b96c4783927d/copy-manifests/0.log"
Sep 07 16:52:44.111043 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:44.111022    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7766cc5489-77jsf_dbee4a3c-3a98-4dfb-a599-b96c4783927d/manager/0.log"
Sep 07 16:52:44.811668 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:44.811641    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5946dbd7c4-qc2z2_136fc2b9-f124-4542-9dd7-2ef278af12fb/maas-api/0.log"
Sep 07 16:52:45.999413 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:45.999379    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-wm5ss_d200e072-bd02-4e20-9681-8f69e53ed62c/manager/0.log"
Sep 07 16:52:46.103153 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:46.103128    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-xvkd2_32482f3f-fc42-4cd4-bbda-e20c251a56ca/manager/0.log"
Sep 07 16:52:46.204576 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:46.204549    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-klrtm_f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd/kuadrant-console-plugin/0.log"
Sep 07 16:52:46.320056 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:46.319982    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-m2w5l_265862c6-e917-439b-a36d-1882e1855bd6/manager/0.log"
Sep 07 16:52:46.445016 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:46.444990    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-ffvz2_3b18d8c6-7df9-42ac-b302-9fff3806eb7a/limitador/0.log"
Sep 07 16:52:47.115850 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:47.115818    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6797cb654d-2cd9b_384721ba-d51f-4bc9-9d85-16b83d60fec1/kube-auth-proxy/0.log"
Sep 07 16:52:47.328349 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:47.328315    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/1.log"
Sep 07 16:52:47.560304 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:47.560278    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:52:47.672343 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:47.672313    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8688c84cc4-cltmt_d86f2ea3-df9d-4ce5-af14-0e49782f295a/router/0.log"
Sep 07 16:52:48.011677 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:48.011646    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8h9hh_4f5751b3-8131-47bb-bf90-6534a76141b6/main/0.log"
Sep 07 16:52:48.675509 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:48.675472    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bjk5h_4b911400-3f32-4d79-a82d-5b14662a16b2/main/0.log"
Sep 07 16:52:53.464563 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.464527    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5v74r/must-gather-vvqvh"]
Sep 07 16:52:53.465040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.465014    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6726ca22-7c51-4be7-81e3-23d3b6821a55" containerName="maas-api"
Sep 07 16:52:53.465040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.465028    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6726ca22-7c51-4be7-81e3-23d3b6821a55" containerName="maas-api"
Sep 07 16:52:53.465130 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.465119    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6726ca22-7c51-4be7-81e3-23d3b6821a55" containerName="maas-api"
Sep 07 16:52:53.469302 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.469285    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.471656 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.471636    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5v74r\"/\"openshift-service-ca.crt\""
Sep 07 16:52:53.472211 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.472190    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5v74r\"/\"default-dockercfg-trgjt\""
Sep 07 16:52:53.472766 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.472727    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5v74r\"/\"kube-root-ca.crt\""
Sep 07 16:52:53.478643 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.478622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5v74r/must-gather-vvqvh"]
Sep 07 16:52:53.570939 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.570906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f008c24f-fc1e-4db8-9478-66bf4c733d36-must-gather-output\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.571122 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.570966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfpqz\" (UniqueName: \"kubernetes.io/projected/f008c24f-fc1e-4db8-9478-66bf4c733d36-kube-api-access-zfpqz\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.672320 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.672288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f008c24f-fc1e-4db8-9478-66bf4c733d36-must-gather-output\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.672472 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.672338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfpqz\" (UniqueName: \"kubernetes.io/projected/f008c24f-fc1e-4db8-9478-66bf4c733d36-kube-api-access-zfpqz\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.672636 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.672617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f008c24f-fc1e-4db8-9478-66bf4c733d36-must-gather-output\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.680283 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.680258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfpqz\" (UniqueName: \"kubernetes.io/projected/f008c24f-fc1e-4db8-9478-66bf4c733d36-kube-api-access-zfpqz\") pod \"must-gather-vvqvh\" (UID: \"f008c24f-fc1e-4db8-9478-66bf4c733d36\") " pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:53.779517 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:53.779436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5v74r/must-gather-vvqvh"
Sep 07 16:52:54.054731 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:54.054703    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5v74r/must-gather-vvqvh"]
Sep 07 16:52:54.055976 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:52:54.055944    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf008c24f_fc1e_4db8_9478_66bf4c733d36.slice/crio-59d04882e6cf823507e9f4a657813f65b687c801ff734d24339d9ec12545e187 WatchSource:0}: Error finding container 59d04882e6cf823507e9f4a657813f65b687c801ff734d24339d9ec12545e187: Status 404 returned error can't find the container with id 59d04882e6cf823507e9f4a657813f65b687c801ff734d24339d9ec12545e187
Sep 07 16:52:54.057812 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:54.057793    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 16:52:54.140591 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:54.140553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5v74r/must-gather-vvqvh" event={"ID":"f008c24f-fc1e-4db8-9478-66bf4c733d36","Type":"ContainerStarted","Data":"59d04882e6cf823507e9f4a657813f65b687c801ff734d24339d9ec12545e187"}
Sep 07 16:52:55.152909 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:55.152799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5v74r/must-gather-vvqvh" event={"ID":"f008c24f-fc1e-4db8-9478-66bf4c733d36","Type":"ContainerStarted","Data":"7cb0077d5d79a44192d2a7bf28b780b7ac93d1493a4e573da3fe8853e88c3db6"}
Sep 07 16:52:55.152909 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:55.152859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5v74r/must-gather-vvqvh" event={"ID":"f008c24f-fc1e-4db8-9478-66bf4c733d36","Type":"ContainerStarted","Data":"f1dc44103bc879e0ae16965bd63c0a88fd1b55ec8d4a6e244e2c30e65394c78d"}
Sep 07 16:52:55.176269 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:55.176204    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5v74r/must-gather-vvqvh" podStartSLOduration=1.434488874 podStartE2EDuration="2.176182448s" totalImagesPullingTime="741.693574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 16:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:52:55.174887774 +0000 UTC m=+2013.819183084" watchObservedRunningTime="2026-09-07 16:52:55.176182448 +0000 UTC m=+2013.820477759"
Sep 07 16:52:56.356657 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:56.356624    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-89lj5_60c3d654-54ef-47cd-bc10-3f1173214ca8/global-pull-secret-syncer/0.log"
Sep 07 16:52:56.569840 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:56.569806    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p5n2k_5d52e7cf-e8e3-469e-b7a7-a5f8b180ecdf/konnectivity-agent/0.log"
Sep 07 16:52:56.623288 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:52:56.623215    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-154.ec2.internal_67dd56507bf34e3ef1ec1fea9b6f36e9/haproxy/0.log"
Sep 07 16:53:01.809375 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:01.809337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-wm5ss_d200e072-bd02-4e20-9681-8f69e53ed62c/manager/0.log"
Sep 07 16:53:01.831610 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:01.831578    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-xvkd2_32482f3f-fc42-4cd4-bbda-e20c251a56ca/manager/0.log"
Sep 07 16:53:01.877983 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:01.877952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-klrtm_f4c6ed83-aeb0-4484-9b87-ad80b4aab4bd/kuadrant-console-plugin/0.log"
Sep 07 16:53:01.926927 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:01.926846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-m2w5l_265862c6-e917-439b-a36d-1882e1855bd6/manager/0.log"
Sep 07 16:53:01.952608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:01.952578    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-ffvz2_3b18d8c6-7df9-42ac-b302-9fff3806eb7a/limitador/0.log"
Sep 07 16:53:03.535833 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:03.535677    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-7whbp_de9dc804-3b56-4941-8d7f-dae66f7ffd14/cluster-monitoring-operator/0.log"
Sep 07 16:53:03.697499 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:03.697471    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6df6_4f9b9d28-f558-4cc7-b00d-2acc1fe86042/node-exporter/0.log"
Sep 07 16:53:03.718609 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:03.718498    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6df6_4f9b9d28-f558-4cc7-b00d-2acc1fe86042/kube-rbac-proxy/0.log"
Sep 07 16:53:03.740478 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:03.740451    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6df6_4f9b9d28-f558-4cc7-b00d-2acc1fe86042/init-textfile/0.log"
Sep 07 16:53:04.195020 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:04.194932    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-thsnm_9fcc16fa-d3b8-4fee-a71c-7604f892b610/prometheus-operator-admission-webhook/0.log"
Sep 07 16:53:05.634403 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:05.634369    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-qc4gc_2aa09c03-bffd-4df4-99b1-3b80887591cc/networking-console-plugin/0.log"
Sep 07 16:53:06.180632 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.180600    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/1.log"
Sep 07 16:53:06.187975 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.187938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bxjnn_d3d62372-86a4-46e4-bb1e-9a4622096b13/console-operator/2.log"
Sep 07 16:53:06.229044 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.229002    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"]
Sep 07 16:53:06.233715 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.233679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.240548 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.240515    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"]
Sep 07 16:53:06.303622 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.303580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-podres\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.303836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.303671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cw4\" (UniqueName: \"kubernetes.io/projected/af4509ca-2209-4f8f-bf83-c91cc2b875d5-kube-api-access-z5cw4\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.303836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.303702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-sys\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.303836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.303782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-lib-modules\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.303836 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.303816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-proc\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405196 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-podres\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cw4\" (UniqueName: \"kubernetes.io/projected/af4509ca-2209-4f8f-bf83-c91cc2b875d5-kube-api-access-z5cw4\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-sys\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-lib-modules\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405380 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-podres\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-proc\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-sys\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-proc\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.405552 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.405493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af4509ca-2209-4f8f-bf83-c91cc2b875d5-lib-modules\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.413270 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.413239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cw4\" (UniqueName: \"kubernetes.io/projected/af4509ca-2209-4f8f-bf83-c91cc2b875d5-kube-api-access-z5cw4\") pod \"perf-node-gather-daemonset-nqxw5\" (UID: \"af4509ca-2209-4f8f-bf83-c91cc2b875d5\") " pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.561181 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.561149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:06.740459 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.740386    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-575ccb6ffc-gzkfk_71f40916-9b6f-407a-86a4-1a798a37c1ef/console/0.log"
Sep 07 16:53:06.774897 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.774854    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-zmd8z_c5c51f07-797b-4b83-b922-d930a9832775/download-server/0.log"
Sep 07 16:53:06.854865 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:06.854827    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"]
Sep 07 16:53:06.860182 ip-10-0-135-154 kubenswrapper[2579]: W0907 16:53:06.860150    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaf4509ca_2209_4f8f_bf83_c91cc2b875d5.slice/crio-e6c01e3cf450beaf549ee991ee3c8d221db2af663f65f8df5ce62f832c907a29 WatchSource:0}: Error finding container e6c01e3cf450beaf549ee991ee3c8d221db2af663f65f8df5ce62f832c907a29: Status 404 returned error can't find the container with id e6c01e3cf450beaf549ee991ee3c8d221db2af663f65f8df5ce62f832c907a29
Sep 07 16:53:07.270514 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:07.270477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5" event={"ID":"af4509ca-2209-4f8f-bf83-c91cc2b875d5","Type":"ContainerStarted","Data":"a1d7307daac1bd9a83157d655b2e3627ab32df0ad39a72d9ed46370cae8fb3e3"}
Sep 07 16:53:07.270723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:07.270539    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5" event={"ID":"af4509ca-2209-4f8f-bf83-c91cc2b875d5","Type":"ContainerStarted","Data":"e6c01e3cf450beaf549ee991ee3c8d221db2af663f65f8df5ce62f832c907a29"}
Sep 07 16:53:07.270723 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:07.270564    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:07.287613 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:07.287543    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5" podStartSLOduration=1.287520579 podStartE2EDuration="1.287520579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 16:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 16:53:07.284195917 +0000 UTC m=+2025.928491226" watchObservedRunningTime="2026-09-07 16:53:07.287520579 +0000 UTC m=+2025.931815889"
Sep 07 16:53:07.419205 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:07.419167    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-lrbmf_7a90ede4-bd7c-4b11-8144-1d87d18fe69e/volume-data-source-validator/0.log"
Sep 07 16:53:08.502633 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:08.502603    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9h7m_6adcc7c9-5071-43f7-8179-31900ce7f8c2/dns/0.log"
Sep 07 16:53:08.519572 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:08.519490    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9h7m_6adcc7c9-5071-43f7-8179-31900ce7f8c2/kube-rbac-proxy/0.log"
Sep 07 16:53:08.579132 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:08.579103    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zhl2d_ec32b97f-faca-4bcb-8733-9b95a727a079/dns-node-resolver/0.log"
Sep 07 16:53:09.138135 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:09.138030    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xcftb_c1746538-b2dc-4bf1-be5e-1bb8fa5bde10/node-ca/0.log"
Sep 07 16:53:10.582434 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:10.582404    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6797cb654d-2cd9b_384721ba-d51f-4bc9-9d85-16b83d60fec1/kube-auth-proxy/0.log"
Sep 07 16:53:10.633040 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:10.633008    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/0.log"
Sep 07 16:53:10.644875 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:10.644847    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/1.log"
Sep 07 16:53:10.727608 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:10.727577    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:53:10.747625 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:10.747597    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8688c84cc4-cltmt_d86f2ea3-df9d-4ce5-af14-0e49782f295a/router/0.log"
Sep 07 16:53:11.582154 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:11.582115    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr97m_a7cbfe08-e923-4162-b279-2f1514acff96/serve-healthcheck-canary/0.log"
Sep 07 16:53:12.043324 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:12.043288    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ncq5c_61dd7487-9e46-4d8b-aba1-7d208e5866fc/insights-operator/0.log"
Sep 07 16:53:12.045291 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:12.045267    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ncq5c_61dd7487-9e46-4d8b-aba1-7d208e5866fc/insights-operator/1.log"
Sep 07 16:53:12.122425 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:12.122392    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj6rz_5d25d52d-ad0c-4ff0-bdea-3da2e656caaa/kube-rbac-proxy/0.log"
Sep 07 16:53:12.139909 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:12.139885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj6rz_5d25d52d-ad0c-4ff0-bdea-3da2e656caaa/exporter/0.log"
Sep 07 16:53:12.161588 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:12.161564    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj6rz_5d25d52d-ad0c-4ff0-bdea-3da2e656caaa/extractor/0.log"
Sep 07 16:53:13.290279 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:13.290249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5v74r/perf-node-gather-daemonset-nqxw5"
Sep 07 16:53:14.189239 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.189209    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cs4vd_2ba65f54-d03d-45ab-8a2b-64e68250d677/manager/0.log"
Sep 07 16:53:14.417791 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.417727    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5qnzr_9d2e8250-239b-415a-ace9-31be31c0a7b9/manager/0.log"
Sep 07 16:53:14.478258 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.478222    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56db7f4c4c-xl2cx_ff017eed-eb10-408f-bc80-3b4c2693d2e7/manager/0.log"
Sep 07 16:53:14.496318 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.496274    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n9xgx_e77947cc-2180-414c-b67c-f5e49a013789/server/0.log"
Sep 07 16:53:14.594457 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.594408    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cwjtb_7af43cc6-f6e9-4c2c-999f-ae32eadaa6d9/manager/0.log"
Sep 07 16:53:14.619121 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.619079    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7766cc5489-77jsf_dbee4a3c-3a98-4dfb-a599-b96c4783927d/manager/0.log"
Sep 07 16:53:14.636737 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:14.636704    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7766cc5489-77jsf_dbee4a3c-3a98-4dfb-a599-b96c4783927d/copy-manifests/0.log"
Sep 07 16:53:17.084605 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:17.084566    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6797cb654d-2cd9b_384721ba-d51f-4bc9-9d85-16b83d60fec1/kube-auth-proxy/0.log"
Sep 07 16:53:17.119795 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:17.119761    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/0.log"
Sep 07 16:53:17.131530 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:17.131508    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/1.log"
Sep 07 16:53:17.198846 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:17.198812    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:53:17.211477 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:17.211449    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8688c84cc4-cltmt_d86f2ea3-df9d-4ce5-af14-0e49782f295a/router/0.log"
Sep 07 16:53:19.216681 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:19.216651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6797cb654d-2cd9b_384721ba-d51f-4bc9-9d85-16b83d60fec1/kube-auth-proxy/0.log"
Sep 07 16:53:19.253658 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:19.253629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/0.log"
Sep 07 16:53:19.268717 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:19.268690    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnd5h_53fd93db-cb25-44db-b969-75c23131cd9d/istio-proxy/1.log"
Sep 07 16:53:19.335151 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:19.335109    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sz5qv_b86846dd-38ae-484c-acb6-5753fa726611/payload-processing/0.log"
Sep 07 16:53:19.352573 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:19.352538    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8688c84cc4-cltmt_d86f2ea3-df9d-4ce5-af14-0e49782f295a/router/0.log"
Sep 07 16:53:21.300137 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:21.300108    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-lstgg_fdc79fad-e548-465c-ab98-afaefef818d5/kube-storage-version-migrator-operator/1.log"
Sep 07 16:53:21.301454 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:21.301427    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-lstgg_fdc79fad-e548-465c-ab98-afaefef818d5/kube-storage-version-migrator-operator/0.log"
Sep 07 16:53:22.260964 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.260825    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6hbb7_d56d8bb8-cab4-41d9-a30e-4ba23a98c374/kube-multus/0.log"
Sep 07 16:53:22.334593 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.334568    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/kube-multus-additional-cni-plugins/0.log"
Sep 07 16:53:22.352124 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.352100    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/egress-router-binary-copy/0.log"
Sep 07 16:53:22.369868 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.369833    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/cni-plugins/0.log"
Sep 07 16:53:22.389614 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.389581    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/bond-cni-plugin/0.log"
Sep 07 16:53:22.407191 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.407169    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/routeoverride-cni/0.log"
Sep 07 16:53:22.424569 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.424525    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/whereabouts-cni-bincopy/0.log"
Sep 07 16:53:22.442347 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.442328    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zdjd_1edc37cf-dc0d-4e09-977f-30a393e7be84/whereabouts-cni/0.log"
Sep 07 16:53:22.853158 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.853127    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zj7m6_d81a54d9-b6b8-4822-b998-e823d5ba34f0/network-metrics-daemon/0.log"
Sep 07 16:53:22.869210 ip-10-0-135-154 kubenswrapper[2579]: I0907 16:53:22.869164    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zj7m6_d81a54d9-b6b8-4822-b998-e823d5ba34f0/kube-rbac-proxy/0.log"