Sep 10 12:09:31.070543 ip-10-0-130-59 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 12:09:31.475475 ip-10-0-130-59 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 12:09:31.475475 ip-10-0-130-59 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 12:09:31.475475 ip-10-0-130-59 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 12:09:31.475475 ip-10-0-130-59 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 12:09:31.475475 ip-10-0-130-59 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 12:09:31.478461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.478400    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 12:09:31.481030 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481015    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481031    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481035    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481038    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481041    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481044    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481047    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481050    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481055    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481059    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481062    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481064    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481067    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481070    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.481066 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481073    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481076    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481079    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481082    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481084    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481088    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481090    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481093    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481095    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481098    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481100    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481102    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481105    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481108    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481111    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481114    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481116    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481119    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481121    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.481416 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481124    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481126    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481128    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481131    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481133    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481136    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481138    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481140    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481143    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481146    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481148    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481152    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481154    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481157    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481159    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481163    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481165    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481168    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481170    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481173    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.482084 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481175    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481178    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481180    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481182    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481185    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481187    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481190    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481192    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481194    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481197    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481199    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481202    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481204    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481206    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481209    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481211    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481214    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481216    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481218    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481221    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.482666 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481223    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481226    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481229    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481232    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481234    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481237    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481239    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481244    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481247    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481249    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481252    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481254    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481257    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481259    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481262    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481264    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481266    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481269    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481271    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481274    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.483145 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481277    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481281    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481296    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481299    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481302    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.481305    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.482998    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483009    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483013    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483016    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483019    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483022    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483034    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483037    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483040    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483043    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483045    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483048    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483051    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.483631 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483053    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483057    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483062    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483065    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483068    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483071    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483074    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483077    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483080    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483083    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483085    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483088    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483093    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483096    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483099    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483102    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483104    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483108    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483112    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.484095 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483115    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483118    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483121    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483124    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483126    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483129    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483132    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483134    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483137    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483139    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483141    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483144    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483146    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483149    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483152    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483154    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483157    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483159    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483162    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483166    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.484649 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483169    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483172    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483174    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483177    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483180    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483182    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483185    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483187    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483190    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483192    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483195    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483197    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483200    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483203    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483205    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483208    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483210    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483213    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483215    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483218    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.485127 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483220    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483222    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483225    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483227    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483230    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483233    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483235    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483238    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483240    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483243    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483245    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483250    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483252    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483255    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483258    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483260    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483262    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483265    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483267    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483270    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.485621 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483272    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483275    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483277    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483280    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483282    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483300    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483303    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483652    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483657    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483659    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483662    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483664    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483667    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483669    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483672    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483675    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483678    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483681    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483683    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483686    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.486096 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483688    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483691    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483693    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483697    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483700    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483702    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483705    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483707    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483710    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483712    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483715    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483718    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483720    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483722    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483725    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483728    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483730    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483733    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483736    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.486609 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483738    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483741    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483743    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483746    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483749    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483752    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483755    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483758    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483760    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483763    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483765    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483767    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483770    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483772    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483775    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483777    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483780    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483784    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483787    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.487064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483790    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483792    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483795    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483797    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483800    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483802    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483805    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483813    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483817    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483821    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483823    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483827    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483830    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483833    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483835    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483838    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483840    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483843    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483845    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483848    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.487522 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483850    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483853    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483855    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483858    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483860    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483863    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483865    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483868    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483870    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483873    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483876    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483879    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483881    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483884    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483887    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483889    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483892    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483895    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483899    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483902    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.488021 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483905    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483907    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483910    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483913    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483916    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483918    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483921    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.483923    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.483991    2551 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484033    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484042    2551 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484046    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484050    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484053    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484057    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484061    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484064    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484067    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484070    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484073    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484076    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484079    2551 flags.go:64] FLAG: --cgroup-root=""
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484081    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 12:09:31.488501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484084    2551 flags.go:64] FLAG: --client-ca-file=""
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484088    2551 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484091    2551 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484094    2551 flags.go:64] FLAG: --cluster-domain=""
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484097    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484100    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484103    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484106    2551 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484110    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484113    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484116    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484119    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484122    2551 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484125    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484127    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484130    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484133    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484137    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484139    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484142    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484145    2551 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484148    2551 flags.go:64] FLAG: --enable-server="true"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484150    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484155    2551 flags.go:64] FLAG: --event-burst="100"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484158    2551 flags.go:64] FLAG: --event-qps="50"
Sep 10 12:09:31.489078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484160    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484163    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484166    2551 flags.go:64] FLAG: --eviction-hard=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484169    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484174    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484177    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484179    2551 flags.go:64] FLAG: --eviction-soft=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484182    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484185    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484188    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484191    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484194    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484196    2551 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484199    2551 flags.go:64] FLAG: --feature-gates=""
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484203    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484206    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484208    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484211    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484214    2551 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484217    2551 flags.go:64] FLAG: --help="false"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484220    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484223    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484226    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 12:09:31.489680 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484229    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484232    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484235    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484238    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484240    2551 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484243    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484246    2551 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484249    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484252    2551 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484254    2551 flags.go:64] FLAG: --kube-reserved=""
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484257    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484259    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484262    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484266    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484269    2551 flags.go:64] FLAG: --lock-file=""
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484272    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484275    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484277    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484282    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484299    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484304    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484309    2551 flags.go:64] FLAG: --logging-format="text"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484312    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484316    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 12:09:31.490250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484319    2551 flags.go:64] FLAG: --manifest-url=""
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484321    2551 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484325    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484328    2551 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484333    2551 flags.go:64] FLAG: --max-pods="110"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484335    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484338    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484341    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484344    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484347    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484349    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484352    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484359    2551 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484362    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484365    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484368    2551 flags.go:64] FLAG: --pod-cidr=""
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484370    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484375    2551 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484378    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484381    2551 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484383    2551 flags.go:64] FLAG: --port="10250"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484388    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484391    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0076e2085266daa74"
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484394    2551 flags.go:64] FLAG: --qos-reserved=""
Sep 10 12:09:31.490822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484397    2551 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484400    2551 flags.go:64] FLAG: --register-node="true"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484403    2551 flags.go:64] FLAG: --register-with-taints=""
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484406    2551 flags.go:64] FLAG: --registry-burst="10"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484411    2551 flags.go:64] FLAG: --registry-qps="5"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484413    2551 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484416    2551 flags.go:64] FLAG: --reserved-memory=""
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484419    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484422    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484425    2551 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484428    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484431    2551 flags.go:64] FLAG: --runonce="false"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484434    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484438    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484441    2551 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484444    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484446    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484449    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484452    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484455    2551 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484458    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484460    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484463    2551 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484466    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484469    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 12:09:31.491419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484472    2551 flags.go:64] FLAG: --system-cgroups=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484474    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484479    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484482    2551 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484486    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484491    2551 flags.go:64] FLAG: --tls-min-version=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484493    2551 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484496    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484499    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484502    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484504    2551 flags.go:64] FLAG: --v="2"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484508    2551 flags.go:64] FLAG: --version="false"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484513    2551 flags.go:64] FLAG: --vmodule=""
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484516    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484519    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484633    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484637    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484640    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484642    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484645    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484649    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484652    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484654    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.492005 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484657    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484660    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484662    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484665    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484667    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484670    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484672    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484675    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484677    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484680    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484683    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484685    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484687    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484691    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484693    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484696    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484699    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484701    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484704    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.492590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484706    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484708    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484711    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484714    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484717    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484719    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484722    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484725    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484727    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484729    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484734    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484738    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484742    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484745    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484748    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484751    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484754    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484756    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484759    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.493039 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484761    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484764    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484767    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484770    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484772    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484775    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484777    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484781    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484784    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484786    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484789    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484791    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484794    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484796    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484799    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484801    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484804    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484807    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484809    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484811    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.493510 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484815    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484817    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484820    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484825    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484828    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484831    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484833    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484836    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484839    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484841    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484843    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484846    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484849    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484851    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484854    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484856    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484858    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484861    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484863    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484867    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.493986 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484869    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484872    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484874    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484877    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484879    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484882    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484884    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484886    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484889    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484891    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484894    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484897    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.484899    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.484904    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.491129    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 12:09:31.494553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.491143    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.491967    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.491978    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492030    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492035    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492038    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492041    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492044    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492047    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492050    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492053    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492055    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492058    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492060    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492065    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492068    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492071    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492074    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492078    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.494908 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492081    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492085    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492088    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492091    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492093    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492096    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492098    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492101    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492103    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492106    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492109    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492111    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492114    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492118    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492121    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492124    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492126    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492129    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492131    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.495387 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492134    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492137    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492140    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492144    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492147    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492149    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492152    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492155    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492157    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492160    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492162    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492165    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492167    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492170    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492173    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492175    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492178    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492180    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492183    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.495869 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492185    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492187    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492190    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492193    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492195    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492197    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492200    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492203    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492207    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492209    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492211    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492214    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492217    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492219    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492222    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492224    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492226    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492229    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492232    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492234    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492237    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.496333 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492239    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492242    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492244    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492247    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492250    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492252    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492255    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492257    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492260    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492262    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492264    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492267    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492269    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492272    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492275    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492277    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492279    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492282    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492296    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.496825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492298    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492301    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492304    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492307    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492309    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.492314    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492415    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492420    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492423    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492425    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492428    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492431    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492433    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492436    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492439    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 12:09:31.497276 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492441    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492445    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492449    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492453    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492456    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492458    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492461    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492463    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492466    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492468    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492471    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492473    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492475    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492478    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492480    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492483    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492485    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492488    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492490    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 12:09:31.497657 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492494    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492496    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492501    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492504    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492507    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492510    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492512    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492515    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492517    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492520    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492522    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492524    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492527    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492530    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492532    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492535    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492537    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492540    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492542    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 12:09:31.498088 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492545    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492547    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492549    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492552    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492554    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492557    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492559    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492561    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492564    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492566    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492568    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492571    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492574    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492576    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492579    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492582    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492584    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492587    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492589    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492592    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 12:09:31.498547 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492595    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492597    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492600    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492603    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492605    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492608    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492610    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492613    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492615    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492618    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492620    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492622    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492625    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492627    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492631    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492634    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492637    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492639    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492642    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492645    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 12:09:31.499034 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492647    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492650    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492652    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492655    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492657    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492660    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492663    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492666    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492668    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492671    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492673    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:31.492675    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.492680    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.492770    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.494696    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 12:09:31.499550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.495515    2551 server.go:1019] "Starting client certificate rotation"
Sep 10 12:09:31.499900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.495610    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 12:09:31.499900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.495646    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 12:09:31.499900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.498188    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 12:09:31.499900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.499221    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 12:09:31.510503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.510486    2551 log.go:25] "Validated CRI v1 runtime API"
Sep 10 12:09:31.514544 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.514528    2551 log.go:25] "Validated CRI v1 image API"
Sep 10 12:09:31.515383 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.515370    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 12:09:31.519377 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.519352    2551 fs.go:135] Filesystem UUIDs: map[2d53d0fb-ed09-456d-90e6-13bf4b187515:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c2bc67df-60bf-4e64-9c63-69a06e6dfc52:/dev/nvme0n1p3]
Sep 10 12:09:31.519377 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.519370    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 12:09:31.524162 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.524061    2551 manager.go:216] Machine: {Timestamp:2026-09-10 12:09:31.522677078 +0000 UTC m=+0.331320904 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3193959 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29a169bff25cdedfd1cde8a43a29a8 SystemUUID:ec29a169-bff2-5cde-dfd1-cde8a43a29a8 BootID:38d446b1-6911-44d6-ae41-55e0139a336d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7b:41:1f:f4:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7b:41:1f:f4:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:0a:75:c1:90:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 12:09:31.524162 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.524160    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 12:09:31.524256 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.524221    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 12:09:31.526826 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.526804    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 12:09:31.526947 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.526829    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 12:09:31.527565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.527554    2551 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 12:09:31.527602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.527568    2551 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 12:09:31.527602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.527575    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 12:09:31.528218 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.528208    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 12:09:31.529031 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.529022    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 12:09:31.529925 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.529911    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 12:09:31.532662 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532651    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 12:09:31.532766 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532757    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 12:09:31.532807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532783    2551 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 12:09:31.532807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532792    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 12:09:31.532807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532806    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 12:09:31.532911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532814    2551 kubelet.go:398] "Adding apiserver pod source"
Sep 10 12:09:31.532911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.532822    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 12:09:31.533172 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.533159    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 12:09:31.533203 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.533187    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 12:09:31.535386 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.535370    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 12:09:31.537349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.537335    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 12:09:31.537418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.537364    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 12:09:31.538332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538322    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538338    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538343    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538348    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538357    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538362    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 12:09:31.538376 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538375    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 12:09:31.538525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538380    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 12:09:31.538525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538386    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 12:09:31.538525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538391    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 12:09:31.538525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538400    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 12:09:31.538525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.538404    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 12:09:31.539202 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.539190    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 12:09:31.539202 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.539199    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 12:09:31.542611 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.542593    2551 server.go:1268] "Started kubelet"
Sep 10 12:09:31.542703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.542661    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 12:09:31.542757 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.542668    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 12:09:31.542757 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.542734    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 12:09:31.543060 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.543034    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 12:09:31.543060 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.543041    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 12:09:31.543193 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.543103    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 12:09:31.543324 ip-10-0-130-59 systemd[1]: Started Kubernetes Kubelet.
Sep 10 12:09:31.543800 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.543783    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 12:09:31.549246 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.549225    2551 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 12:09:31.549718 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.549687    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 12:09:31.551174 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.551155    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 12:09:31.551599 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.551579    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 12:09:31.551682 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.551666    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.552393 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552364    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 12:09:31.552393 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552387    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 12:09:31.552393 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552364    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 12:09:31.552555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552399    2551 factory.go:55] Registering systemd factory
Sep 10 12:09:31.552555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552414    2551 factory.go:223] Registration of the systemd container factory successfully
Sep 10 12:09:31.552555 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.551459    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46052fd043c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.542570044 +0000 UTC m=+0.351213851,LastTimestamp:2026-09-10 12:09:31.542570044 +0000 UTC m=+0.351213851,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.552555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552461    2551 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 12:09:31.552555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552469    2551 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 12:09:31.552701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552600    2551 factory.go:153] Registering CRI-O factory
Sep 10 12:09:31.552701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552609    2551 factory.go:223] Registration of the crio container factory successfully
Sep 10 12:09:31.552701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552643    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 12:09:31.552701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552658    2551 factory.go:103] Registering Raw factory
Sep 10 12:09:31.552701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552667    2551 manager.go:1196] Started watching for new ooms in manager
Sep 10 12:09:31.552927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.552917    2551 manager.go:349] Starting recovery of all containers
Sep 10 12:09:31.561240 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.561110    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 12:09:31.561240 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.561176    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 12:09:31.561775 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.561757    2551 manager.go:354] Recovery completed
Sep 10 12:09:31.566277 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.566260    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.568533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.568517    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.568603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.568543    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.568603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.568558    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.569071 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569056    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 12:09:31.569119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569072    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 12:09:31.569119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569085    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 12:09:31.569119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569061    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.569761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569748    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.569806 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569771    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.569806 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.569780    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.570675 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.570616    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054892c7b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.568532603 +0000 UTC m=+0.377176417,LastTimestamp:2026-09-10 12:09:31.568532603 +0000 UTC m=+0.377176417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.571268 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.571255    2551 policy_none.go:49] "None policy: Start"
Sep 10 12:09:31.571338 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.571271    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 12:09:31.571338 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.571280    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 12:09:31.573240 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.573226    2551 policy_none.go:47] "Start"
Sep 10 12:09:31.578790 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.578723    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054897287  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.568550535 +0000 UTC m=+0.377194341,LastTimestamp:2026-09-10 12:09:31.568550535 +0000 UTC m=+0.377194341,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.581200 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.580895    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054899eba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.56856185 +0000 UTC m=+0.377205655,LastTimestamp:2026-09-10 12:09:31.56856185 +0000 UTC m=+0.377205655,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.581654 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.581636    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxtxj"
Sep 10 12:09:31.587473 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.587395    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-59.ec2.internal.18d3f46054892c7b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054892c7b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.568532603 +0000 UTC m=+0.377176417,LastTimestamp:2026-09-10 12:09:31.569761239 +0000 UTC m=+0.378405045,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.591801 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.591774    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxtxj"
Sep 10 12:09:31.593627 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.593573    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-59.ec2.internal.18d3f46054897287\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054897287  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.568550535 +0000 UTC m=+0.377194341,LastTimestamp:2026-09-10 12:09:31.569775487 +0000 UTC m=+0.378419293,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.597404 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.597270    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-59.ec2.internal.18d3f46054899eba\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d3f46054899eba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 12:09:31.56856185 +0000 UTC m=+0.377205655,LastTimestamp:2026-09-10 12:09:31.569783649 +0000 UTC m=+0.378427455,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}"
Sep 10 12:09:31.600826 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.600812    2551 manager.go:337] "Starting Device Plugin manager"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.600842    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.600850    2551 server.go:87] "Starting device plugin registration server"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.601107    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.601117    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.601226    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.601375    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.601385    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.601814    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 12:09:31.624250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.601865    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.687118 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.687093    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 12:09:31.688143 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.688129    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 12:09:31.688198 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.688169    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 12:09:31.688198 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.688189    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 12:09:31.688278 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.688221    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 12:09:31.690487 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.690468    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 12:09:31.701218 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.701194    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.701947 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.701932    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.702013 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.701957    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.702013 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.701966    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.702013 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.701983    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.710570 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.710555    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.710632 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.710577    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-59.ec2.internal\": node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.742275 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.742220    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.789018 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.788999    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"]
Sep 10 12:09:31.789085 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.789050    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.790267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.790254    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.790336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.790281    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.790336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.790312    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.792407 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.792394    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.792533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.792521    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.792573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.792547    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.793029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793015    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.793063 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793041    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.793063 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793050    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.793117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793018    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.793117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793111    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.793171 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.793119    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.794944 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.794930    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.795035 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.794951    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:31.795577 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.795562    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:31.795644 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.795586    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:31.795644 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.795598    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:31.808823 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.808805    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.812972 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.812960    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.843266 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.843251    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.854744 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.854725    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.854805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.854751    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.854805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.854773    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.943690 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:31.943673    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:31.954971 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.954955    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.955019 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.954976    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.955061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.955021    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.955061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.955044    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.955119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.955056    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:31.955119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:31.955081    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:32.044385 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:32.044347    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:32.110799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:32.110778    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:32.115856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:32.115835    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:32.145420 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:32.145399    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:32.245877 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:32.245858    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.374494 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:33.374456    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.475006 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:33.474987    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.523485 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.523466    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 12:09:33.523762 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.523576    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 12:09:33.575191 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:33.575175    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.624255 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.624227    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 12:04:32 +0000 UTC" deadline="2028-06-21 18:21:51.533964454 +0000 UTC"
Sep 10 12:09:33.624306 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.624263    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15606h12m17.909715812s"
Sep 10 12:09:33.650904 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.650868    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 12:09:33.676199 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:33.676180    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.721713 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.721695    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 12:09:33.751583 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.751568    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 12:09:33.752419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.752357    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 12:09:33.752419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.752392    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 12:09:33.752419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.752402    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID"
Sep 10 12:09:33.776244 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:33.776217    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found"
Sep 10 12:09:33.867854 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.867832    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 12:09:33.880963 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.880945    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:33.897986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.897969    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 12:09:33.899001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.898987    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal"
Sep 10 12:09:33.915807 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:33.915782    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667ccdd66cb316e2e58a7c2fdcf49d11.slice/crio-b627fe02d5c35c17d339b0b42b5ae7a840fe75f50e27f72459ea31e74d8c3e82 WatchSource:0}: Error finding container b627fe02d5c35c17d339b0b42b5ae7a840fe75f50e27f72459ea31e74d8c3e82: Status 404 returned error can't find the container with id b627fe02d5c35c17d339b0b42b5ae7a840fe75f50e27f72459ea31e74d8c3e82
Sep 10 12:09:33.917158 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.917146    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 12:09:33.921464 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:33.921449    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:09:33.958109 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:33.958088    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48adfbd41af55e45ba98e8777ebfc093.slice/crio-ec04fddc064c7dea3e854f3a021527f461470f497f1652f770cf2ac042ecd42b WatchSource:0}: Error finding container ec04fddc064c7dea3e854f3a021527f461470f497f1652f770cf2ac042ecd42b: Status 404 returned error can't find the container with id ec04fddc064c7dea3e854f3a021527f461470f497f1652f770cf2ac042ecd42b
Sep 10 12:09:34.508898 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.508881    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 12:09:34.562426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.562409    2551 apiserver.go:52] "Watching apiserver"
Sep 10 12:09:34.572350 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.572330    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 12:09:34.573493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.573475    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wcjsq","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc","openshift-multus/multus-additional-cni-plugins-q6vsb","openshift-multus/multus-ljz2q","openshift-multus/network-metrics-daemon-frgs7","openshift-network-diagnostics/network-check-target-bn48x","kube-system/global-pull-secret-syncer-x9z4k","openshift-cluster-node-tuning-operator/tuned-p5hbm","openshift-dns/node-resolver-569pc","openshift-image-registry/node-ca-9w79l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","openshift-network-operator/iptables-alerter-tdg27","openshift-ovn-kubernetes/ovnkube-node-nj4hc"]
Sep 10 12:09:34.576063 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.576049    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:34.576119 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.576103    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:34.578107 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.578089    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.579632 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.579617    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 12:09:34.580128 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.580113    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.580592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.580574    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.580592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.580583    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.580896 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.580874    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s7b5q\""
Sep 10 12:09:34.580988 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.580876    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 12:09:34.582260 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.582243    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.584703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.584333    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.584703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.584405    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.584703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.584408    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 12:09:34.584703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.584666    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pmxqr\""
Sep 10 12:09:34.584703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.584666    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.586525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.586508    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.586597 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.586571    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:34.586838 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.586824    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 12:09:34.586946 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.586932    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r9rk5\""
Sep 10 12:09:34.587328 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.587314    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.587403 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.587346    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 12:09:34.587457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.587424    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 12:09:34.588061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.588045    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.588132 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.588061    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 12:09:34.588710 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.588697    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.589052 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.589037    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-29jdx\""
Sep 10 12:09:34.590978 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.590961    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.592441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592422    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592510 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592554 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592509    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-iptables-alerter-script\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.592554 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592536    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-system-cni-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592558    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:34.592624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592574    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-host-slash\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.592624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592588    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx7c9\" (UniqueName: \"kubernetes.io/projected/5f70ccae-062e-4a1d-8191-20e3b9e49c33-kube-api-access-mx7c9\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.592731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592626    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cnibin\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592663    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592688    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5kvb\" (UniqueName: \"kubernetes.io/projected/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-kube-api-access-t5kvb\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.592731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592708    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f70ccae-062e-4a1d-8191-20e3b9e49c33-host\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.592731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592722    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f70ccae-062e-4a1d-8191-20e3b9e49c33-serviceca\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.592892 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592749    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glnj\" (UniqueName: \"kubernetes.io/projected/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-kube-api-access-5glnj\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.592892 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.592773    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.593080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.593067    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.594842 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.594827    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.595737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.595436    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.595737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.595480    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.595737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.595507    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.595737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.595546    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.595924 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.595827    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 12:09:34.596502 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.596483    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-brggr\""
Sep 10 12:09:34.596556 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.596486    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 12:09:34.596556 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.596531    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8fdbx\""
Sep 10 12:09:34.596679 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.596486    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6frm2\""
Sep 10 12:09:34.597763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.597744    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.597856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.597824    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 12:09:34.597856 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.597817    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:34.599296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.599279    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.600028 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.600012    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.602985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.600988    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 12:09:34.602985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.601033    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.602985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.601301    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mz2xh\""
Sep 10 12:09:34.602985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.602579    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 12:09:34.602985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.602637    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 12:09:34.603547 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.603530    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 12:09:34.604450 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.604435    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 12:09:34.604585 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.604553    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 12:09:34.604585 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.604568    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-crvvv\""
Sep 10 12:09:34.604695 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.604612    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 12:09:34.647502 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.647481    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpb2r"
Sep 10 12:09:34.655968 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.655932    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpb2r"
Sep 10 12:09:34.681201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.681181    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 12:09:34.692982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.692961    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-daemon-config\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.692993    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-var-lib-kubelet\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.693072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693018    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5fc\" (UniqueName: \"kubernetes.io/projected/4e3641de-4676-4b50-8ac8-976f1c6646c0-kube-api-access-2h5fc\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.693072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693042    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-node-log\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693063    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693082    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-hostroot\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693103    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-conf-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693132    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-multus-certs\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693155    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-etc-kubernetes\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693176    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-host\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-kubelet-config\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693245    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-ovn\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693270 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693263    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-tmp\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693284    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-sys-fs\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693320    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-config\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693367    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f70ccae-062e-4a1d-8191-20e3b9e49c33-serviceca\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693424    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glnj\" (UniqueName: \"kubernetes.io/projected/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-kube-api-access-5glnj\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693470    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61c014f8-7c3a-4f98-8602-79e3286d8604-konnectivity-ca\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693504    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-registration-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.693642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693623    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693663    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-netd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693744    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-lib-modules\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693769    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693777    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f70ccae-062e-4a1d-8191-20e3b9e49c33-serviceca\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693811    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-script-lib\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693863    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-iptables-alerter-script\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693914    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzd2d\" (UniqueName: \"kubernetes.io/projected/66058bce-9cbb-462f-be4e-82a10bd47ebf-kube-api-access-lzd2d\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693942    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-run\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693965    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d415e4-d474-4315-a005-8f14ab9940f6-ovn-node-metrics-cert\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.693999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.693995    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-host-slash\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694019    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-systemd\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694064    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-env-overrides\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694100    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-system-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694123    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-bin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694146    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysconfig\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694170    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-socket-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694192    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-device-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694205    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-host-slash\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694170    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694260    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694312    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694334    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psqvh\" (UniqueName: \"kubernetes.io/projected/41918e83-ab1b-499f-ade6-4494a5fdba5e-kube-api-access-psqvh\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694341    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-iptables-alerter-script\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.694503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694358    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89qch\" (UniqueName: \"kubernetes.io/projected/955654cc-3b3c-4320-bb69-76ba876dc342-kube-api-access-89qch\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694364    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694411    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61c014f8-7c3a-4f98-8602-79e3286d8604-agent-certs\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694435    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-bin\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694469    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bxlq\" (UniqueName: \"kubernetes.io/projected/d4d415e4-d474-4315-a005-8f14ab9940f6-kube-api-access-7bxlq\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694494    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-conf\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694523    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694548    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-slash\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694598    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-etc-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694649    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-log-socket\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694683    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694717    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx7c9\" (UniqueName: \"kubernetes.io/projected/5f70ccae-062e-4a1d-8191-20e3b9e49c33-kube-api-access-mx7c9\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694780    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-cnibin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694926    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-systemd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694968    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f70ccae-062e-4a1d-8191-20e3b9e49c33-host\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.694992    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/955654cc-3b3c-4320-bb69-76ba876dc342-hosts-file\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.695288 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695016    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-dbus\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695038    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695064    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695066    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f70ccae-062e-4a1d-8191-20e3b9e49c33-host\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695119    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695152    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-kubelet\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695182    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-systemd-units\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkklz\" (UniqueName: \"kubernetes.io/projected/de4e1d71-1163-422a-b8aa-5653e3678ac5-kube-api-access-bkklz\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695251    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-system-cni-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695275    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-cni-binary-copy\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695295    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-modprobe-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695302    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-system-cni-dir\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695314    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-sys\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695341    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-tuned\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695364    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cnibin\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695402    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-kubernetes\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695423    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/955654cc-3b3c-4320-bb69-76ba876dc342-tmp-dir\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.696016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695425    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-cnibin\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695455    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-netns\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5kvb\" (UniqueName: \"kubernetes.io/projected/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-kube-api-access-t5kvb\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695510    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-socket-dir-parent\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695532    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-netns\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695547    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695554    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-multus\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695576    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-var-lib-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695606    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695630    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-k8s-cni-cncf-io\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.696803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.695653    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-kubelet\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.708434 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.708400    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 12:09:34.709737 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.709500    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:34.709737 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.709523    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:34.709737 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.709535    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:34.709737 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.709606    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:35.209586738 +0000 UTC m=+2.990144195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:34.712987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.712853    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx7c9\" (UniqueName: \"kubernetes.io/projected/5f70ccae-062e-4a1d-8191-20e3b9e49c33-kube-api-access-mx7c9\") pod \"node-ca-9w79l\" (UID: \"5f70ccae-062e-4a1d-8191-20e3b9e49c33\") " pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.713082 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.712867    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5kvb\" (UniqueName: \"kubernetes.io/projected/6cf1ab4e-a872-43cb-a670-e85a3ff20cc5-kube-api-access-t5kvb\") pod \"iptables-alerter-tdg27\" (UID: \"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5\") " pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.713658 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.713642    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glnj\" (UniqueName: \"kubernetes.io/projected/e8011ac5-40aa-4a34-9730-b3ba5fdf0035-kube-api-access-5glnj\") pod \"multus-additional-cni-plugins-q6vsb\" (UID: \"e8011ac5-40aa-4a34-9730-b3ba5fdf0035\") " pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.721522 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.721478    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"48adfbd41af55e45ba98e8777ebfc093","Type":"ContainerStarted","Data":"ec04fddc064c7dea3e854f3a021527f461470f497f1652f770cf2ac042ecd42b"}
Sep 10 12:09:34.723415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.723387    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerStarted","Data":"b627fe02d5c35c17d339b0b42b5ae7a840fe75f50e27f72459ea31e74d8c3e82"}
Sep 10 12:09:34.796693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796643    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61c014f8-7c3a-4f98-8602-79e3286d8604-konnectivity-ca\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.796693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-registration-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.796693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796713    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-netd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796748    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-lib-modules\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796772    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796796    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-script-lib\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796817    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzd2d\" (UniqueName: \"kubernetes.io/projected/66058bce-9cbb-462f-be4e-82a10bd47ebf-kube-api-access-lzd2d\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796837    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-run\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.796872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796859    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d415e4-d474-4315-a005-8f14ab9940f6-ovn-node-metrics-cert\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796884    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-systemd\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796906    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-env-overrides\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796927    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-system-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796949    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-bin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.796975    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysconfig\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797021    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-socket-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797033    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-registration-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797044    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-device-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797070    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797079    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-run\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797091    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-netd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797106    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.797161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797145    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psqvh\" (UniqueName: \"kubernetes.io/projected/41918e83-ab1b-499f-ade6-4494a5fdba5e-kube-api-access-psqvh\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.797676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797167    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89qch\" (UniqueName: \"kubernetes.io/projected/955654cc-3b3c-4320-bb69-76ba876dc342-kube-api-access-89qch\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.797676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797175    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-lib-modules\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.798508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.797207    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.798508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798311    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61c014f8-7c3a-4f98-8602-79e3286d8604-konnectivity-ca\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.798508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798399    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysconfig\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.798508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798454    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-system-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.798508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798481    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-bin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798582    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798639    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-socket-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798647    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-device-dir\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798718    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798733    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.798787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798752    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-script-lib\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798804    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-systemd\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798904    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61c014f8-7c3a-4f98-8602-79e3286d8604-agent-certs\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798908    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-env-overrides\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798938    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-bin\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.798970    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bxlq\" (UniqueName: \"kubernetes.io/projected/d4d415e4-d474-4315-a005-8f14ab9940f6-kube-api-access-7bxlq\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799000    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-conf\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799023    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-cni-bin\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799033    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799131    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799144    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-sysctl-conf\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799169    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-slash\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799193    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-etc-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799221    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-log-socket\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799283    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-slash\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799332    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-etc-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799342    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-log-socket\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799293    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-cnibin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799416    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-systemd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799422    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-cnibin\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799475    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-systemd\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799519    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/955654cc-3b3c-4320-bb69-76ba876dc342-hosts-file\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799550    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-dbus\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799577    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799610    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799639    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-kubelet\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799700    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-dbus\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799706    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-kubelet\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.799756 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799739    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-systemd-units\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799773    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkklz\" (UniqueName: \"kubernetes.io/projected/de4e1d71-1163-422a-b8aa-5653e3678ac5-kube-api-access-bkklz\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799780    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/955654cc-3b3c-4320-bb69-76ba876dc342-hosts-file\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.799798    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799801    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-cni-binary-copy\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.799880    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.799889    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-systemd-units\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.800032    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:09:35.300013896 +0000 UTC m=+3.080571363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:34.800059    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:35.300041937 +0000 UTC m=+3.080599396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800077    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-modprobe-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800107    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-sys\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800135    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-tuned\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800158    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-kubernetes\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800220    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/955654cc-3b3c-4320-bb69-76ba876dc342-tmp-dir\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800250    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-netns\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800283    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-socket-dir-parent\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800307    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-netns\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800335    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-multus\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800364    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-var-lib-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800411    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800445    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-k8s-cni-cncf-io\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800508    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-kubelet\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800538    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-daemon-config\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800538    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-cni-binary-copy\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800615    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-modprobe-d\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800627    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-kubernetes\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800667    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-var-lib-kubelet\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800697    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5fc\" (UniqueName: \"kubernetes.io/projected/4e3641de-4676-4b50-8ac8-976f1c6646c0-kube-api-access-2h5fc\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800725    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-node-log\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.800749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800753    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800780    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-hostroot\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800802    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-conf-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800842    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-multus-certs\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800875    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-etc-kubernetes\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800892    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/955654cc-3b3c-4320-bb69-76ba876dc342-tmp-dir\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800902    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-host\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800930    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-kubelet-config\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800955    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-ovn\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800982    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-tmp\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.800994    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-socket-dir-parent\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801010    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-sys-fs\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801053    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-config\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801067    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e3641de-4676-4b50-8ac8-976f1c6646c0-sys-fs\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801140    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-hostroot\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801159    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-netns\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801191    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-cni-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801210    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-cni-multus\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.801467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801235    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-conf-dir\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801259    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-var-lib-openvswitch\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801397    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-var-lib-kubelet\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801444    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-k8s-cni-cncf-io\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801480    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-etc-kubernetes\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-run-ovn\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801557    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801607    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-sys\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801670    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-var-lib-kubelet\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801850    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-host-run-netns\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801904    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41918e83-ab1b-499f-ade6-4494a5fdba5e-host\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801946    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d415e4-d474-4315-a005-8f14ab9940f6-node-log\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.801967    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d415e4-d474-4315-a005-8f14ab9940f6-ovnkube-config\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.802027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66058bce-9cbb-462f-be4e-82a10bd47ebf-host-run-multus-certs\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.802210    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d86fe67c-88f7-4a0f-b640-c1150019d3fc-kubelet-config\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:34.802862 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.802300    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66058bce-9cbb-462f-be4e-82a10bd47ebf-multus-daemon-config\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.802911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.802871    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61c014f8-7c3a-4f98-8602-79e3286d8604-agent-certs\") pod \"konnectivity-agent-wcjsq\" (UID: \"61c014f8-7c3a-4f98-8602-79e3286d8604\") " pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.804056 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.804034    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d415e4-d474-4315-a005-8f14ab9940f6-ovn-node-metrics-cert\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.804421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.804403    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-etc-tuned\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.806114 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.806093    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41918e83-ab1b-499f-ade6-4494a5fdba5e-tmp\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.807704 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.807682    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzd2d\" (UniqueName: \"kubernetes.io/projected/66058bce-9cbb-462f-be4e-82a10bd47ebf-kube-api-access-lzd2d\") pod \"multus-ljz2q\" (UID: \"66058bce-9cbb-462f-be4e-82a10bd47ebf\") " pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.814513 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.814448    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89qch\" (UniqueName: \"kubernetes.io/projected/955654cc-3b3c-4320-bb69-76ba876dc342-kube-api-access-89qch\") pod \"node-resolver-569pc\" (UID: \"955654cc-3b3c-4320-bb69-76ba876dc342\") " pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.815274 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.815231    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkklz\" (UniqueName: \"kubernetes.io/projected/de4e1d71-1163-422a-b8aa-5653e3678ac5-kube-api-access-bkklz\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:34.819761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.819740    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bxlq\" (UniqueName: \"kubernetes.io/projected/d4d415e4-d474-4315-a005-8f14ab9940f6-kube-api-access-7bxlq\") pod \"ovnkube-node-nj4hc\" (UID: \"d4d415e4-d474-4315-a005-8f14ab9940f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.819856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.819779    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5fc\" (UniqueName: \"kubernetes.io/projected/4e3641de-4676-4b50-8ac8-976f1c6646c0-kube-api-access-2h5fc\") pod \"aws-ebs-csi-driver-node-dlhmc\" (UID: \"4e3641de-4676-4b50-8ac8-976f1c6646c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.821197 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.821176    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psqvh\" (UniqueName: \"kubernetes.io/projected/41918e83-ab1b-499f-ade6-4494a5fdba5e-kube-api-access-psqvh\") pod \"tuned-p5hbm\" (UID: \"41918e83-ab1b-499f-ade6-4494a5fdba5e\") " pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.888058 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.888038    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tdg27"
Sep 10 12:09:34.894442 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.894348    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9w79l"
Sep 10 12:09:34.899755 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.899732    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf1ab4e_a872_43cb_a670_e85a3ff20cc5.slice/crio-cf09a1cde1896b9d0eb5ba25b3e326cb213df531e24da116ddd03d51f997d852 WatchSource:0}: Error finding container cf09a1cde1896b9d0eb5ba25b3e326cb213df531e24da116ddd03d51f997d852: Status 404 returned error can't find the container with id cf09a1cde1896b9d0eb5ba25b3e326cb213df531e24da116ddd03d51f997d852
Sep 10 12:09:34.900872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.900799    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6vsb"
Sep 10 12:09:34.904475 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.904452    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f70ccae_062e_4a1d_8191_20e3b9e49c33.slice/crio-487e44f3f179a6aee6e865493bf66b417247f595a279fa888c88b384872808d1 WatchSource:0}: Error finding container 487e44f3f179a6aee6e865493bf66b417247f595a279fa888c88b384872808d1: Status 404 returned error can't find the container with id 487e44f3f179a6aee6e865493bf66b417247f595a279fa888c88b384872808d1
Sep 10 12:09:34.906296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.906277    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljz2q"
Sep 10 12:09:34.909113 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.909094    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8011ac5_40aa_4a34_9730_b3ba5fdf0035.slice/crio-1e9c78cf1b469fa157c53b5b7c4f6272ad76d7e4fa3590dfbb20d7751180df01 WatchSource:0}: Error finding container 1e9c78cf1b469fa157c53b5b7c4f6272ad76d7e4fa3590dfbb20d7751180df01: Status 404 returned error can't find the container with id 1e9c78cf1b469fa157c53b5b7c4f6272ad76d7e4fa3590dfbb20d7751180df01
Sep 10 12:09:34.910708 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.910625    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p5hbm"
Sep 10 12:09:34.915133 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.915108    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66058bce_9cbb_462f_be4e_82a10bd47ebf.slice/crio-fd6c0319d2e5f73c8a92ce1ba8e342f52b4d0184082e2e95a9a53fee62637f3a WatchSource:0}: Error finding container fd6c0319d2e5f73c8a92ce1ba8e342f52b4d0184082e2e95a9a53fee62637f3a: Status 404 returned error can't find the container with id fd6c0319d2e5f73c8a92ce1ba8e342f52b4d0184082e2e95a9a53fee62637f3a
Sep 10 12:09:34.915648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.915631    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-569pc"
Sep 10 12:09:34.919763 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.919733    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41918e83_ab1b_499f_ade6_4494a5fdba5e.slice/crio-80e299808fff726810b27f11ce4cc512daf391d910b2c61b079da18fb5178444 WatchSource:0}: Error finding container 80e299808fff726810b27f11ce4cc512daf391d910b2c61b079da18fb5178444: Status 404 returned error can't find the container with id 80e299808fff726810b27f11ce4cc512daf391d910b2c61b079da18fb5178444
Sep 10 12:09:34.920644 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.920608    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:34.924771 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.924749    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955654cc_3b3c_4320_bb69_76ba876dc342.slice/crio-a5fadd800c86b4db89a5dc2a09963a31e583d20bc1cb0b2f8f047d10aac574ab WatchSource:0}: Error finding container a5fadd800c86b4db89a5dc2a09963a31e583d20bc1cb0b2f8f047d10aac574ab: Status 404 returned error can't find the container with id a5fadd800c86b4db89a5dc2a09963a31e583d20bc1cb0b2f8f047d10aac574ab
Sep 10 12:09:34.925948 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.925827    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc"
Sep 10 12:09:34.929778 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.929760    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c014f8_7c3a_4f98_8602_79e3286d8604.slice/crio-4db679c5fa0f62b433e235d2ef7b0970eacbdda6dc59f63d97e82d777a84c991 WatchSource:0}: Error finding container 4db679c5fa0f62b433e235d2ef7b0970eacbdda6dc59f63d97e82d777a84c991: Status 404 returned error can't find the container with id 4db679c5fa0f62b433e235d2ef7b0970eacbdda6dc59f63d97e82d777a84c991
Sep 10 12:09:34.930573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:34.930554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:34.937514 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.937492    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3641de_4676_4b50_8ac8_976f1c6646c0.slice/crio-ad0c3039e37e5194e7db50ba863983925516d9c26452638da77b02d8e87c98da WatchSource:0}: Error finding container ad0c3039e37e5194e7db50ba863983925516d9c26452638da77b02d8e87c98da: Status 404 returned error can't find the container with id ad0c3039e37e5194e7db50ba863983925516d9c26452638da77b02d8e87c98da
Sep 10 12:09:34.940286 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:09:34.940239    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d415e4_d474_4315_a005_8f14ab9940f6.slice/crio-85c7a1e6afe0e87fe953dd43b7a3a6cdd0c08c270f6f7732f5dcc66a8a5217d0 WatchSource:0}: Error finding container 85c7a1e6afe0e87fe953dd43b7a3a6cdd0c08c270f6f7732f5dcc66a8a5217d0: Status 404 returned error can't find the container with id 85c7a1e6afe0e87fe953dd43b7a3a6cdd0c08c270f6f7732f5dcc66a8a5217d0
Sep 10 12:09:35.304902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.304874    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:35.305078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.304908    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:35.305078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.304926    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:35.305184 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305093    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:35.305184 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305111    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:35.305184 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305120    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:35.305184 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305161    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:36.305148919 +0000 UTC m=+4.085706370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:35.305365 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305027    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:35.305365 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305254    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:36.305238865 +0000 UTC m=+4.085796316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:35.305365 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305326    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:35.305594 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.305392    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:09:36.305362055 +0000 UTC m=+4.085919519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:35.659104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.659016    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:04:34 +0000 UTC" deadline="2028-03-04 10:30:23.27541603 +0000 UTC"
Sep 10 12:09:35.659104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.659053    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12982h20m47.61636736s"
Sep 10 12:09:35.717488 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.717463    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:35.717635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:35.717589    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:35.726181 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.726098    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerStarted","Data":"1e9c78cf1b469fa157c53b5b7c4f6272ad76d7e4fa3590dfbb20d7751180df01"}
Sep 10 12:09:35.727487 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.727454    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9w79l" event={"ID":"5f70ccae-062e-4a1d-8191-20e3b9e49c33","Type":"ContainerStarted","Data":"487e44f3f179a6aee6e865493bf66b417247f595a279fa888c88b384872808d1"}
Sep 10 12:09:35.728779 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.728743    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wcjsq" event={"ID":"61c014f8-7c3a-4f98-8602-79e3286d8604","Type":"ContainerStarted","Data":"4db679c5fa0f62b433e235d2ef7b0970eacbdda6dc59f63d97e82d777a84c991"}
Sep 10 12:09:35.729942 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.729920    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tdg27" event={"ID":"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5","Type":"ContainerStarted","Data":"cf09a1cde1896b9d0eb5ba25b3e326cb213df531e24da116ddd03d51f997d852"}
Sep 10 12:09:35.731233 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.731209    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"85c7a1e6afe0e87fe953dd43b7a3a6cdd0c08c270f6f7732f5dcc66a8a5217d0"}
Sep 10 12:09:35.732639 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.732610    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc" event={"ID":"4e3641de-4676-4b50-8ac8-976f1c6646c0","Type":"ContainerStarted","Data":"ad0c3039e37e5194e7db50ba863983925516d9c26452638da77b02d8e87c98da"}
Sep 10 12:09:35.733730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.733709    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-569pc" event={"ID":"955654cc-3b3c-4320-bb69-76ba876dc342","Type":"ContainerStarted","Data":"a5fadd800c86b4db89a5dc2a09963a31e583d20bc1cb0b2f8f047d10aac574ab"}
Sep 10 12:09:35.735865 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.735846    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p5hbm" event={"ID":"41918e83-ab1b-499f-ade6-4494a5fdba5e","Type":"ContainerStarted","Data":"80e299808fff726810b27f11ce4cc512daf391d910b2c61b079da18fb5178444"}
Sep 10 12:09:35.737094 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:35.737073    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljz2q" event={"ID":"66058bce-9cbb-462f-be4e-82a10bd47ebf","Type":"ContainerStarted","Data":"fd6c0319d2e5f73c8a92ce1ba8e342f52b4d0184082e2e95a9a53fee62637f3a"}
Sep 10 12:09:36.311851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.311818    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:36.312051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.311865    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:36.312051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.311891    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:36.312051 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312019    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:36.312220 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312075    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:38.312058734 +0000 UTC m=+6.092616193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:36.312568 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312551    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:36.312646 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312573    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:36.312646 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312587    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:36.312646 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312633    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:38.312616486 +0000 UTC m=+6.093173943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:36.312800 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312694    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:36.312800 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.312726    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:09:38.312714236 +0000 UTC m=+6.093271694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:36.659582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.659543    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:04:34 +0000 UTC" deadline="2028-02-28 05:19:22.500420485 +0000 UTC"
Sep 10 12:09:36.659582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.659583    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12857h9m45.840841579s"
Sep 10 12:09:36.719667 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.719645    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:36.719785 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.719760    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:36.720125 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:36.720106    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:36.720214 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:36.720194    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:37.717982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:37.717469    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:37.717982 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:37.717598    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:37.763701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:37.763662    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"48adfbd41af55e45ba98e8777ebfc093","Type":"ContainerStarted","Data":"89196658d6ca9d4f666d7af8a649a8955559190cd852352f9834ebbfa0420ea4"}
Sep 10 12:09:37.769776 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:37.769747    2551 generic.go:356] "Generic (PLEG): container finished" podID="667ccdd66cb316e2e58a7c2fdcf49d11" containerID="2b5673df2ccb62354ee9a444ff9aec0b930920e61efb11aba5c5f4a004e93866" exitCode=0
Sep 10 12:09:37.769885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:37.769798    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerDied","Data":"2b5673df2ccb62354ee9a444ff9aec0b930920e61efb11aba5c5f4a004e93866"}
Sep 10 12:09:37.799040 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:37.798238    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" podStartSLOduration=4.798220372 podStartE2EDuration="4.798220372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:37.781153052 +0000 UTC m=+5.561710526" watchObservedRunningTime="2026-09-10 12:09:37.798220372 +0000 UTC m=+5.578777845"
Sep 10 12:09:38.329761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:38.329729    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:38.329761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:38.329769    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:38.329986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:38.329791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:38.329986 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.329912    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:38.329986 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.329968    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:42.329950896 +0000 UTC m=+10.110508349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:38.330398 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330354    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:38.330398 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330397    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:38.330554 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330409    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:38.330554 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330454    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:42.330438287 +0000 UTC m=+10.110995739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:38.330554 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330530    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:38.330713 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.330567    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:09:42.330555322 +0000 UTC m=+10.111112774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:38.719290 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:38.719217    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:38.719727 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.719387    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:38.720235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:38.720213    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:38.720361 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:38.720339    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:39.717028 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:39.716995    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:39.717198 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:39.717137    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:40.717005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:40.716932    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:40.717417 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:40.717051    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:40.717417 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:40.717299    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:40.717533 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:40.717420    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:41.716978 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:41.716911    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:41.717121 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:41.717027    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:42.363468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:42.363347    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:42.363468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:42.363413    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:42.363485    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363510    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363583    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:09:50.363563033 +0000 UTC m=+18.144120491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363590    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363608    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363620    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363665    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:50.363650731 +0000 UTC m=+18.144208196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:42.363705 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363705    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:42.364111 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.363766    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:09:50.363755852 +0000 UTC m=+18.144313303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:42.717123 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:42.717037    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:42.717569 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.717164    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:42.718352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:42.718326    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:42.718466 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:42.718434    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:43.717089 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:43.717055    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:43.717258 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:43.717183    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:44.716818 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:44.716782    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:44.716962 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:44.716898    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:44.716962 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:44.716952    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:44.717052 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:44.717038    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:45.716723 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:45.716693    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:45.717149 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:45.716829    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:46.717570 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:46.717535    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:46.717991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:46.717549    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:46.717991 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:46.717646    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:46.717991 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:46.717726    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:47.717003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:47.716971    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:47.717160 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:47.717087    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:48.717002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:48.716929    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:48.717441 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:48.717048    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:48.717441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:48.717123    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:48.717441 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:48.717230    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:49.716468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:49.716434    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:49.716649 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:49.716554    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:50.427693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:50.427649    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:50.427693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:50.427695    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:50.427721    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427803    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427816    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427832    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427838    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427844    2551 projected.go:196] Error preparing data for projected volume kube-api-access-dx5pj for pod openshift-network-diagnostics/network-check-target-bn48x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427869    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.42785177 +0000 UTC m=+34.208409228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427888    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj podName:71b09d5b-dc17-40b2-9f1c-1730ad7f88a5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.427878552 +0000 UTC m=+34.208436004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dx5pj" (UniqueName: "kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj") pod "network-check-target-bn48x" (UID: "71b09d5b-dc17-40b2-9f1c-1730ad7f88a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 12:09:50.428250 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.427903    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.427895245 +0000 UTC m=+34.208452696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:09:50.717122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:50.717042    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:50.717277 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.717163    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:50.717277 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:50.717253    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:50.717398 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:50.717358    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:51.717183 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:51.717150    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:51.717596 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:51.717257    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:52.718043 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.717718    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:52.718659 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.717832    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:52.718659 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:52.718143    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:52.718659 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:52.718203    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:52.799313 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.799288    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:09:52.799741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.799717    2551 generic.go:356] "Generic (PLEG): container finished" podID="d4d415e4-d474-4315-a005-8f14ab9940f6" containerID="c08cb0483ae59c12e56aad10697ca2b5aa77ab3d72f59eef34d499ae7a0fb3cc" exitCode=1
Sep 10 12:09:52.799823 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.799786    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"eb35c233440e3de6d8a2a6c134262c0928a3fb8f5f0d5b9496e066ccfdaa88ed"}
Sep 10 12:09:52.799882 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.799872    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerDied","Data":"c08cb0483ae59c12e56aad10697ca2b5aa77ab3d72f59eef34d499ae7a0fb3cc"}
Sep 10 12:09:52.799944 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.799929    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"2e65b8d178dc38ce36fb5459841f88c70e1d6109e52db4d59c5981b058b707c5"}
Sep 10 12:09:52.801283 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.801257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc" event={"ID":"4e3641de-4676-4b50-8ac8-976f1c6646c0","Type":"ContainerStarted","Data":"2d88a81f0c01cbc78b51bb08cc3cc21cb403393bf314d5b4a2e03b5ae5b28ae2"}
Sep 10 12:09:52.802638 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.802617    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-569pc" event={"ID":"955654cc-3b3c-4320-bb69-76ba876dc342","Type":"ContainerStarted","Data":"c9b9cdb73876e701dabc9b8693b431c1f1a39e41a066d7cc2a7bcb9bc9f1293e"}
Sep 10 12:09:52.804034 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.804017    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p5hbm" event={"ID":"41918e83-ab1b-499f-ade6-4494a5fdba5e","Type":"ContainerStarted","Data":"aa3b5b501ad0e3309374064ee6a39d636db52d81c40cd3c526fa3672c586dcdb"}
Sep 10 12:09:52.805421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.805401    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljz2q" event={"ID":"66058bce-9cbb-462f-be4e-82a10bd47ebf","Type":"ContainerStarted","Data":"c7e32ed8c8f8c17af519eaff7d41ccec1e0f7a38e25e72844c9e006498235bb6"}
Sep 10 12:09:52.806852 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.806833    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="f44174309d5bfea33ffbd9d8fb3fc840e6475607623be790d8361ddb9d40ff87" exitCode=0
Sep 10 12:09:52.806963 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.806910    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"f44174309d5bfea33ffbd9d8fb3fc840e6475607623be790d8361ddb9d40ff87"}
Sep 10 12:09:52.808850 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.808648    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9w79l" event={"ID":"5f70ccae-062e-4a1d-8191-20e3b9e49c33","Type":"ContainerStarted","Data":"8dc907ba401b37cd56661ca0648948737b6b7e8bde5333b9b705b148ee5ebc93"}
Sep 10 12:09:52.810726 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.810704    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerStarted","Data":"a7c45ea43cc40eb0e1f5cf337590a189409ccc863fe4b5f2f9c6bd084b6ecb22"}
Sep 10 12:09:52.812347 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.812314    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wcjsq" event={"ID":"61c014f8-7c3a-4f98-8602-79e3286d8604","Type":"ContainerStarted","Data":"cea9df97f6dce921fa40b3a0605819db1ad620db4d62eda1471e441b9df3dd0d"}
Sep 10 12:09:52.864504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.864432    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-569pc" podStartSLOduration=3.394608472 podStartE2EDuration="20.864412074s" totalImagesPullingTime="17.469803602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:52.829669764 +0000 UTC m=+20.610227263" watchObservedRunningTime="2026-09-10 12:09:52.864412074 +0000 UTC m=+20.644969550"
Sep 10 12:09:52.891233 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.891198    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" podStartSLOduration=19.891184198 podStartE2EDuration="19.891184198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:52.8645533 +0000 UTC m=+20.645110765" watchObservedRunningTime="2026-09-10 12:09:52.891184198 +0000 UTC m=+20.671741670"
Sep 10 12:09:52.915653 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.915613    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ljz2q" podStartSLOduration=3.306839147 podStartE2EDuration="20.915595664s" totalImagesPullingTime="17.608756517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:52.890940061 +0000 UTC m=+20.671497551" watchObservedRunningTime="2026-09-10 12:09:52.915595664 +0000 UTC m=+20.696153139"
Sep 10 12:09:52.936959 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:52.936927    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p5hbm" podStartSLOduration=3.461373678 podStartE2EDuration="20.936915388s" totalImagesPullingTime="17.47554171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:52.915356084 +0000 UTC m=+20.695913577" watchObservedRunningTime="2026-09-10 12:09:52.936915388 +0000 UTC m=+20.717472863"
Sep 10 12:09:53.012765 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.012725    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9w79l" podStartSLOduration=3.937255387 podStartE2EDuration="21.012712838s" totalImagesPullingTime="17.075457451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:52.937108053 +0000 UTC m=+20.717665529" watchObservedRunningTime="2026-09-10 12:09:53.012712838 +0000 UTC m=+20.793270310"
Sep 10 12:09:53.307081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.307051    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:53.307678 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.307653    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:53.338909 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.338868    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wcjsq" podStartSLOduration=6.856520996 podStartE2EDuration="20.338855955s" totalImagesPullingTime="13.482334959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:53.056686086 +0000 UTC m=+20.837243560" watchObservedRunningTime="2026-09-10 12:09:53.338855955 +0000 UTC m=+21.119413426"
Sep 10 12:09:53.717144 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.717115    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:53.717310 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:53.717236    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:53.816552 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.816516    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tdg27" event={"ID":"6cf1ab4e-a872-43cb-a670-e85a3ff20cc5","Type":"ContainerStarted","Data":"0dcf808bc047f3eab25037d9b0b8f35b4d60a79d3bc8be2ba639fc9e91cb5686"}
Sep 10 12:09:53.819750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.819730    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:09:53.820169 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.820144    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"124cd835e4bf4858bb6e41d266d90dd29a708b3fc235231a4c94b818e4638c32"}
Sep 10 12:09:53.820256 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.820224    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"2637b61670e268b5dcce14d82222421ff8dd4669cd10d09c695d585d5a7cdff1"}
Sep 10 12:09:53.820880 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.820856    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:53.820978 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.820899    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"e8a633d8210dad9b8949bc7aa64e8ced29e84b18c763564616fdb9bc9fa8d964"}
Sep 10 12:09:53.821038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.821021    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wcjsq"
Sep 10 12:09:53.842740 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:53.842693    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tdg27" podStartSLOduration=4.348451811 podStartE2EDuration="21.842674813s" totalImagesPullingTime="17.494223002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:53.842234535 +0000 UTC m=+21.622792033" watchObservedRunningTime="2026-09-10 12:09:53.842674813 +0000 UTC m=+21.623232287"
Sep 10 12:09:54.049739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.049598    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 12:09:54.642520 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.642429    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T12:09:54.049734487Z","UUID":"587812a5-5547-474b-b5d3-63602c34e264","Handler":null,"Name":"","Endpoint":""}
Sep 10 12:09:54.644120 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.644096    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 12:09:54.644238 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.644128    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 12:09:54.716901 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.716866    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:54.717042 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:54.716979    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:54.717108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.717039    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:54.717175 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:54.717149    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:54.824176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:54.824142    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc" event={"ID":"4e3641de-4676-4b50-8ac8-976f1c6646c0","Type":"ContainerStarted","Data":"4c998a3498edc3169d6f41e7ed2b7aaf28937969c5bfe2d9f46200afefa8ae4c"}
Sep 10 12:09:55.717429 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:55.717353    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:55.717576 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:55.717473    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:55.829561 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:55.829527    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:09:55.829997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:55.829966    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"3970c1e8e0eafa07a3531272f448a05bcdd562a7633099ea1c392d89c66e97b5"}
Sep 10 12:09:55.832416 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:55.832362    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc" event={"ID":"4e3641de-4676-4b50-8ac8-976f1c6646c0","Type":"ContainerStarted","Data":"962cecd85bad2078b211648b1c208e710ea0566f694954e916e443af610548b4"}
Sep 10 12:09:55.858224 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:55.858170    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlhmc" podStartSLOduration=2.973998641 podStartE2EDuration="22.858153847s" totalImagesPullingTime="19.884155206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:55.858108843 +0000 UTC m=+23.638666354" watchObservedRunningTime="2026-09-10 12:09:55.858153847 +0000 UTC m=+23.638711321"
Sep 10 12:09:56.717213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:56.717188    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:56.717461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:56.717190    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:56.717461 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:56.717292    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:56.717461 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:56.717363    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:57.716557 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.716399    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:57.716902 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:57.716640    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:57.839841 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.839819    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:09:57.840196 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.840175    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"5711cbe2848bd32ffa580516bc22a06a880d5f53e300b28ed6c81adb3a9d4b5c"}
Sep 10 12:09:57.840455 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.840409    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:57.840597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.840480    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:57.840597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.840569    2551 scope.go:117] "RemoveContainer" containerID="c08cb0483ae59c12e56aad10697ca2b5aa77ab3d72f59eef34d499ae7a0fb3cc"
Sep 10 12:09:57.842354 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.842332    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="4d01d79a86532c5908bb184430a4b3b458dc83be4bff857108ac9bcedc065baa" exitCode=0
Sep 10 12:09:57.842460 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.842390    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"4d01d79a86532c5908bb184430a4b3b458dc83be4bff857108ac9bcedc065baa"}
Sep 10 12:09:57.857177 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:57.857156    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:58.717099 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.717071    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:58.717849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.717071    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:58.717849 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:58.717179    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:58.717849 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:58.717243    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:58.849131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.849109    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:09:58.849596 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.849561    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" event={"ID":"d4d415e4-d474-4315-a005-8f14ab9940f6","Type":"ContainerStarted","Data":"f3479f7ebee7176d44c34947f88b254d7a61eebc4e81be8e128226dad3e089aa"}
Sep 10 12:09:58.849947 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.849928    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:58.865920 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.865895    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:09:58.885151 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:58.885101    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc" podStartSLOduration=8.416215885 podStartE2EDuration="25.885084193s" totalImagesPullingTime="17.468868308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:58.883137978 +0000 UTC m=+26.663695469" watchObservedRunningTime="2026-09-10 12:09:58.885084193 +0000 UTC m=+26.665641667"
Sep 10 12:09:59.167512 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.167485    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x9z4k"]
Sep 10 12:09:59.167621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.167608    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:09:59.167704 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:59.167683    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:09:59.170537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.170516    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bn48x"]
Sep 10 12:09:59.170636 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.170592    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:09:59.170692 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:59.170650    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:09:59.173444 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.173423    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-frgs7"]
Sep 10 12:09:59.173537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.173520    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:09:59.173644 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:09:59.173624    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:09:59.852976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.852945    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="8685e3ce1d867c7d612cd14bac1237946967d007a7b95b1ea0b2f448a7028c67" exitCode=0
Sep 10 12:09:59.853459 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:09:59.853020    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"8685e3ce1d867c7d612cd14bac1237946967d007a7b95b1ea0b2f448a7028c67"}
Sep 10 12:10:00.717109 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:00.717083    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:00.717334 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:00.717090    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:00.717334 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:00.717175    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:10:00.717334 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:00.717267    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:10:00.717334 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:00.717288    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:00.717553 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:00.717391    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:10:01.859836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:01.859667    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="87d8f578fc6d9fa1197cbe90d410647a04ce9074f6f1b58b9e8950834b72b06a" exitCode=0
Sep 10 12:10:01.860160 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:01.859746    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"87d8f578fc6d9fa1197cbe90d410647a04ce9074f6f1b58b9e8950834b72b06a"}
Sep 10 12:10:02.718138 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:02.718111    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:02.718329 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:02.718201    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:10:02.718329 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:02.718236    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:02.718329 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:02.718234    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:02.718502 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:02.718341    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:10:02.718502 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:02.718410    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:10:04.717479 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:04.717445    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:04.717909 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:04.717446    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:04.717909 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:04.717573    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:04.717909 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:04.717572    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x9z4k" podUID="d86fe67c-88f7-4a0f-b640-c1150019d3fc"
Sep 10 12:10:04.717909 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:04.717656    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bn48x" podUID="71b09d5b-dc17-40b2-9f1c-1730ad7f88a5"
Sep 10 12:10:04.717909 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:04.717755    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frgs7" podUID="de4e1d71-1163-422a-b8aa-5653e3678ac5"
Sep 10 12:10:05.433729 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.433700    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeReady"
Sep 10 12:10:05.433875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.433821    2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 12:10:05.500457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.500430    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:10:05.530623 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.530596    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zlph5"]
Sep 10 12:10:05.530755 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.530725    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.546458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.546437    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"]
Sep 10 12:10:05.546573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.546543    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.560859 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.560812    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 12:10:05.561529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561347    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.561529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561347    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 12:10:05.561529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561388    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.561529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561439    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 12:10:05.561529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561504    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 12:10:05.562010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.561987    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-k9xcx\""
Sep 10 12:10:05.562113 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.562017    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vctr\""
Sep 10 12:10:05.563290 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.563271    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"]
Sep 10 12:10:05.563458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.563438    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"
Sep 10 12:10:05.576273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.576243    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 12:10:05.576542 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.576520    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.576780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.576758    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.576988 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.576973    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zcs2q\""
Sep 10 12:10:05.585324 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.585300    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.585324 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.585316    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b6767dd58-vrxv6"]
Sep 10 12:10:05.591919 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.591899    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 12:10:05.592902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.592884    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fmqj2\""
Sep 10 12:10:05.593898 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.593866    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.593999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.593901    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.602250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.602228    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 12:10:05.604967 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.604951    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 12:10:05.607351 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.607332    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-tgczq"]
Sep 10 12:10:05.607471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.607453    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.609724 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.609707    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 12:10:05.609805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.609707    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 12:10:05.609986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.609966    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 12:10:05.610323 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.610279    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.615897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.615649    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 12:10:05.615897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.615686    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.620505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.619450    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hrltp\""
Sep 10 12:10:05.623525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.623507    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"]
Sep 10 12:10:05.623688 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.623662    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.641881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.641858    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zlph5"]
Sep 10 12:10:05.641970 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.641895    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lrtv7"]
Sep 10 12:10:05.642091 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642072    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.642827 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642809    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-serving-cert\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.642928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642835    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j727\" (UniqueName: \"kubernetes.io/projected/0f1a4986-32cd-45f9-86b6-3eb5410b7069-kube-api-access-9j727\") pod \"volume-data-source-validator-695c895bd6-ttt4x\" (UID: \"0f1a4986-32cd-45f9-86b6-3eb5410b7069\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"
Sep 10 12:10:05.642928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642860    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-trusted-ca\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.642928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642886    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndhr\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.642928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642913    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642941    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642958    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.642979    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.643005    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dgzw\" (UniqueName: \"kubernetes.io/projected/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-kube-api-access-8dgzw\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.643136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.643081    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643456 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.643188    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-config\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.643456 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.643241    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.643456 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.643267    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.646061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.646042    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 12:10:05.646061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.646040    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7c527\""
Sep 10 12:10:05.646200 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.646138    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.646200 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.646049    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 12:10:05.653114 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.653094    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"]
Sep 10 12:10:05.653218 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.653131    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:10:05.653218 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.653152    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"]
Sep 10 12:10:05.653427 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.653405    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.672708 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.672681    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.673102 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.672895    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 12:10:05.673102 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.672945    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-56gbm\""
Sep 10 12:10:05.673102 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.672894    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.673102 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.673038    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.673588 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.673569    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 12:10:05.675461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.675444    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-m646p"]
Sep 10 12:10:05.675587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.675569    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.692846 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.692827    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.692957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.692942    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.693820 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693805    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b6767dd58-vrxv6"]
Sep 10 12:10:05.693906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693829    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"]
Sep 10 12:10:05.693906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-tgczq"]
Sep 10 12:10:05.693906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693858    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lrtv7"]
Sep 10 12:10:05.693906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693881    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b8kql"]
Sep 10 12:10:05.694171 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.693918    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-m646p"
Sep 10 12:10:05.697730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.697582    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7696c\""
Sep 10 12:10:05.697730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.697606    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.697730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.697638    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rqlwd\""
Sep 10 12:10:05.697928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.697895    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698137    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698144    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698164    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rv9dp\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698189    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698216    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698236    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 12:10:05.698271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.698147    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.708238 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.708219    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-29d4k"]
Sep 10 12:10:05.708339 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.708323    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.710637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.710619    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 12:10:05.710858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.710838    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 12:10:05.711125 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.711105    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7jv8\""
Sep 10 12:10:05.730805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.730788    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-m646p"]
Sep 10 12:10:05.731077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.730813    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"]
Sep 10 12:10:05.731077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.730829    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"]
Sep 10 12:10:05.731077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.730850    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"]
Sep 10 12:10:05.731077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.730899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.735209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.735189    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gxdf\""
Sep 10 12:10:05.735773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.735728    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 12:10:05.735876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.735862    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 12:10:05.744254 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744235    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-stats-auth\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.744341 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744271    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-config\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.744341 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744296    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744341 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744321    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82423b91-84fc-44e5-b3a3-5f3c23ee4452-cert\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.744497 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744370    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744548 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.744534    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:05.744597 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.744551    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:05.744633 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.744600    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.244581075 +0000 UTC m=+34.025138531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:05.744685 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744631    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744682    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.744780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744739    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5bnz\" (UniqueName: \"kubernetes.io/projected/32ad9dc7-8703-4e37-8708-b28b17491b29-kube-api-access-r5bnz\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.744824 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744800    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndhr\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744870 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744828    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-snapshots\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.744870 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744851    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744870    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.744943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744890    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-tmp\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.744943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744919    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dgzw\" (UniqueName: \"kubernetes.io/projected/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-kube-api-access-8dgzw\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.745074 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.744999    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-config\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.745124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745111    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9ccr\" (UniqueName: \"kubernetes.io/projected/82423b91-84fc-44e5-b3a3-5f3c23ee4452-kube-api-access-s9ccr\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.745170 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745146    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/886da443-9220-47dc-9222-7b1aca3a6e78-config\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.745170 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745162    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwsc9\" (UniqueName: \"kubernetes.io/projected/886da443-9220-47dc-9222-7b1aca3a6e78-kube-api-access-bwsc9\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.745268 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745193    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40af8ac6-a203-4ac5-8e54-64baac884dda-serving-cert\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.745268 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745211    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x29zc\" (UniqueName: \"kubernetes.io/projected/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-kube-api-access-x29zc\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.745268 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.745432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745296    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-serving-cert\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.745432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745326    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j727\" (UniqueName: \"kubernetes.io/projected/0f1a4986-32cd-45f9-86b6-3eb5410b7069-kube-api-access-9j727\") pod \"volume-data-source-validator-695c895bd6-ttt4x\" (UID: \"0f1a4986-32cd-45f9-86b6-3eb5410b7069\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"
Sep 10 12:10:05.745432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745356    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-trusted-ca\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.745432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745397    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-default-certificate\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745430    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745455    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745480    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/886da443-9220-47dc-9222-7b1aca3a6e78-serving-cert\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745537    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745569    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.745625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745597    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.745878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745654    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2mvh\" (UniqueName: \"kubernetes.io/projected/40af8ac6-a203-4ac5-8e54-64baac884dda-kube-api-access-w2mvh\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.745878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745707    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.745878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745731    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.745878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745750    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5lkr\" (UniqueName: \"kubernetes.io/projected/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-kube-api-access-p5lkr\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.745878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.745772    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.746474 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.746099    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-trusted-ca\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.746474 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.746303    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.746474 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.746364    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.749405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.749364    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-serving-cert\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.749405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.749395    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.749541 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.749430    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.752963 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.752946    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-29d4k"]
Sep 10 12:10:05.753051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.752981    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b8kql"]
Sep 10 12:10:05.753051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.753000    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"]
Sep 10 12:10:05.753140 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.753058    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.756480 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.756462    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 12:10:05.757900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.757876    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 12:10:05.757986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.757945    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:05.758054 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.757981    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 12:10:05.758808 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.758788    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5ffgw\""
Sep 10 12:10:05.766816 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.766650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndhr\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.766816 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.766754    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dgzw\" (UniqueName: \"kubernetes.io/projected/2d29e9a0-2c23-42b5-80b1-2a961a66e48e-kube-api-access-8dgzw\") pod \"console-operator-85b454874b-zlph5\" (UID: \"2d29e9a0-2c23-42b5-80b1-2a961a66e48e\") " pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.767046 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.767026    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j727\" (UniqueName: \"kubernetes.io/projected/0f1a4986-32cd-45f9-86b6-3eb5410b7069-kube-api-access-9j727\") pod \"volume-data-source-validator-695c895bd6-ttt4x\" (UID: \"0f1a4986-32cd-45f9-86b6-3eb5410b7069\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"
Sep 10 12:10:05.767566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.767544    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:05.846637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846608    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.846637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846642    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-default-certificate\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.846835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846661    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.846835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.846835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846744    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/886da443-9220-47dc-9222-7b1aca3a6e78-serving-cert\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.846835 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.846777    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.346759136 +0000 UTC m=+34.127316595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:05.846835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846815    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846841    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846867    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2mvh\" (UniqueName: \"kubernetes.io/projected/40af8ac6-a203-4ac5-8e54-64baac884dda-kube-api-access-w2mvh\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846901    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j89m7\" (UniqueName: \"kubernetes.io/projected/15caee24-9bdb-490a-a632-ba9b4ecbede4-kube-api-access-j89m7\") pod \"network-check-source-685456655f-m646p\" (UID: \"15caee24-9bdb-490a-a632-ba9b4ecbede4\") " pod="openshift-network-diagnostics/network-check-source-685456655f-m646p"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846933    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846958    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5lkr\" (UniqueName: \"kubernetes.io/projected/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-kube-api-access-p5lkr\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.846985    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.847080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847012    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-stats-auth\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.847240    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847259    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.847306    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.347288681 +0000 UTC m=+34.127846136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847328    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82423b91-84fc-44e5-b3a3-5f3c23ee4452-cert\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847389    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847418    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5bnz\" (UniqueName: \"kubernetes.io/projected/32ad9dc7-8703-4e37-8708-b28b17491b29-kube-api-access-r5bnz\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847448    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08296382-68e4-4fa4-837b-47b37de1c881-nginx-conf\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847476    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-snapshots\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.847505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847505    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-tmp\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847530    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsj4\" (UniqueName: \"kubernetes.io/projected/5208d4e0-3d95-49be-80f3-f3e681fc63b5-kube-api-access-gnsj4\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847558    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb8de4-a0c3-4250-bca8-44c17293d423-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847580    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/90fb8de4-a0c3-4250-bca8-44c17293d423-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847626    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9ccr\" (UniqueName: \"kubernetes.io/projected/82423b91-84fc-44e5-b3a3-5f3c23ee4452-kube-api-access-s9ccr\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847649    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kt5v\" (UniqueName: \"kubernetes.io/projected/90fb8de4-a0c3-4250-bca8-44c17293d423-kube-api-access-9kt5v\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847687    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/886da443-9220-47dc-9222-7b1aca3a6e78-config\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847689    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847711    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwsc9\" (UniqueName: \"kubernetes.io/projected/886da443-9220-47dc-9222-7b1aca3a6e78-kube-api-access-bwsc9\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.847735    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208d4e0-3d95-49be-80f3-f3e681fc63b5-tmp-dir\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.847826    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:05.847940 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.847866    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.347853314 +0000 UTC m=+34.128410786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:05.848419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848063    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40af8ac6-a203-4ac5-8e54-64baac884dda-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.848419 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.848148    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 12:10:05.848419 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.848220    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls podName:32ad9dc7-8703-4e37-8708-b28b17491b29 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.348204153 +0000 UTC m=+34.128761624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-5t45d" (UID: "32ad9dc7-8703-4e37-8708-b28b17491b29") : secret "samples-operator-tls" not found
Sep 10 12:10:05.848546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848413    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/886da443-9220-47dc-9222-7b1aca3a6e78-config\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.848546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848496    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40af8ac6-a203-4ac5-8e54-64baac884dda-serving-cert\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.848546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848536    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x29zc\" (UniqueName: \"kubernetes.io/projected/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-kube-api-access-x29zc\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.848687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848593    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-snapshots\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.848687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848656    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40af8ac6-a203-4ac5-8e54-64baac884dda-tmp\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.848982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848884    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5208d4e0-3d95-49be-80f3-f3e681fc63b5-config-volume\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.848982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.848919    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.850051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.850008    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-default-certificate\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.850707 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.850683    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-stats-auth\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.850957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.850935    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40af8ac6-a203-4ac5-8e54-64baac884dda-serving-cert\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.851120 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.851090    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/886da443-9220-47dc-9222-7b1aca3a6e78-serving-cert\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.851277 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.851259    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82423b91-84fc-44e5-b3a3-5f3c23ee4452-cert\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.855492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.855447    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wcjsq_61c014f8-7c3a-4f98-8602-79e3286d8604/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:05.857345 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.857325    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:05.866074 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.866032    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5lkr\" (UniqueName: \"kubernetes.io/projected/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-kube-api-access-p5lkr\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:05.866074 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.866063    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwsc9\" (UniqueName: \"kubernetes.io/projected/886da443-9220-47dc-9222-7b1aca3a6e78-kube-api-access-bwsc9\") pod \"service-ca-operator-859f94c4b-zx2lv\" (UID: \"886da443-9220-47dc-9222-7b1aca3a6e78\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.867727 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.867671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x29zc\" (UniqueName: \"kubernetes.io/projected/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-kube-api-access-x29zc\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:05.868103 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.868085    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9ccr\" (UniqueName: \"kubernetes.io/projected/82423b91-84fc-44e5-b3a3-5f3c23ee4452-kube-api-access-s9ccr\") pod \"ingress-canary-lrtv7\" (UID: \"82423b91-84fc-44e5-b3a3-5f3c23ee4452\") " pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.868852 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.868830    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5bnz\" (UniqueName: \"kubernetes.io/projected/32ad9dc7-8703-4e37-8708-b28b17491b29-kube-api-access-r5bnz\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:05.872702 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.872684    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"
Sep 10 12:10:05.873009 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.872992    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2mvh\" (UniqueName: \"kubernetes.io/projected/40af8ac6-a203-4ac5-8e54-64baac884dda-kube-api-access-w2mvh\") pod \"insights-operator-76bcd6ccf5-tgczq\" (UID: \"40af8ac6-a203-4ac5-8e54-64baac884dda\") " pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.932545 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.932520    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq"
Sep 10 12:10:05.949762 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.949741    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5208d4e0-3d95-49be-80f3-f3e681fc63b5-config-volume\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.949869 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.949775    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.949929 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.949890    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:05.949982 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.949940    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.449921825 +0000 UTC m=+34.230479294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:05.949982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.949959    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.950100 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950019    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j89m7\" (UniqueName: \"kubernetes.io/projected/15caee24-9bdb-490a-a632-ba9b4ecbede4-kube-api-access-j89m7\") pod \"network-check-source-685456655f-m646p\" (UID: \"15caee24-9bdb-490a-a632-ba9b4ecbede4\") " pod="openshift-network-diagnostics/network-check-source-685456655f-m646p"
Sep 10 12:10:05.950100 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950089    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08296382-68e4-4fa4-837b-47b37de1c881-nginx-conf\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.950209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950122    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsj4\" (UniqueName: \"kubernetes.io/projected/5208d4e0-3d95-49be-80f3-f3e681fc63b5-kube-api-access-gnsj4\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.950209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950147    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb8de4-a0c3-4250-bca8-44c17293d423-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.950209 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.950159    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:05.950209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950171    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/90fb8de4-a0c3-4250-bca8-44c17293d423-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.950436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950216    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kt5v\" (UniqueName: \"kubernetes.io/projected/90fb8de4-a0c3-4250-bca8-44c17293d423-kube-api-access-9kt5v\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.950436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950253    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208d4e0-3d95-49be-80f3-f3e681fc63b5-tmp-dir\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.950436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950386    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5208d4e0-3d95-49be-80f3-f3e681fc63b5-config-volume\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.950595 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208d4e0-3d95-49be-80f3-f3e681fc63b5-tmp-dir\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.950645 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:05.950595    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:06.450581017 +0000 UTC m=+34.231138473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:05.950986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.950852    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/90fb8de4-a0c3-4250-bca8-44c17293d423-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.951217 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.951171    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08296382-68e4-4fa4-837b-47b37de1c881-nginx-conf\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:05.952782 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.952756    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"
Sep 10 12:10:05.952934 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.952912    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb8de4-a0c3-4250-bca8-44c17293d423-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.961800 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.961782    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lrtv7"
Sep 10 12:10:05.968382 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.968348    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsj4\" (UniqueName: \"kubernetes.io/projected/5208d4e0-3d95-49be-80f3-f3e681fc63b5-kube-api-access-gnsj4\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:05.982027 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.982006    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kt5v\" (UniqueName: \"kubernetes.io/projected/90fb8de4-a0c3-4250-bca8-44c17293d423-kube-api-access-9kt5v\") pod \"kube-storage-version-migrator-operator-8d7dd4675-jxnnl\" (UID: \"90fb8de4-a0c3-4250-bca8-44c17293d423\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:05.987784 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:05.987761    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j89m7\" (UniqueName: \"kubernetes.io/projected/15caee24-9bdb-490a-a632-ba9b4ecbede4-kube-api-access-j89m7\") pod \"network-check-source-685456655f-m646p\" (UID: \"15caee24-9bdb-490a-a632-ba9b4ecbede4\") " pod="openshift-network-diagnostics/network-check-source-685456655f-m646p"
Sep 10 12:10:06.002989 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.002973    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-m646p"
Sep 10 12:10:06.053289 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.053263    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wcjsq_61c014f8-7c3a-4f98-8602-79e3286d8604/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:06.077952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.077935    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"
Sep 10 12:10:06.252726 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.252653    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:06.252860 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.252820    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:06.252860 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.252844    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:06.252946 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.252908    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.252889043 +0000 UTC m=+35.033446508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:06.353524 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.353488    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:06.353684 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.353635    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:06.353684 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353652    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:06.353799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.353694    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:06.353799 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353711    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.353695273 +0000 UTC m=+35.134252724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:06.353799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.353761    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:06.353799 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353784    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:06.353998 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353790    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.353775021 +0000 UTC m=+35.134332481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:06.353998 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353835    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.353824093 +0000 UTC m=+35.134381560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:06.353998 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353851    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 12:10:06.353998 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.353895    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls podName:32ad9dc7-8703-4e37-8708-b28b17491b29 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.353882642 +0000 UTC m=+35.134440118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-5t45d" (UID: "32ad9dc7-8703-4e37-8708-b28b17491b29") : secret "samples-operator-tls" not found
Sep 10 12:10:06.454254 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.454226    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:06.454254 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.454259    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:06.454448 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.454276    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:06.454448 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.454326    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:06.454448 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454388    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:10:06.454448 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.454400    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:06.454448 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454440    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs podName:de4e1d71-1163-422a-b8aa-5653e3678ac5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:38.454424652 +0000 UTC m=+66.234982123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs") pod "network-metrics-daemon-frgs7" (UID: "de4e1d71-1163-422a-b8aa-5653e3678ac5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454465    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454487    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454487    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454520    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.454504324 +0000 UTC m=+35.235061776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454536    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret podName:d86fe67c-88f7-4a0f-b640-c1150019d3fc nodeName:}" failed. No retries permitted until 2026-09-10 12:10:38.454527676 +0000 UTC m=+66.235085131 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret") pod "global-pull-secret-syncer-x9z4k" (UID: "d86fe67c-88f7-4a0f-b640-c1150019d3fc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 12:10:06.454635 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:06.454549    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:07.454541949 +0000 UTC m=+35.235099406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:06.456692 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.456676    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx5pj\" (UniqueName: \"kubernetes.io/projected/71b09d5b-dc17-40b2-9f1c-1730ad7f88a5-kube-api-access-dx5pj\") pod \"network-check-target-bn48x\" (UID: \"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5\") " pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:06.720550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.720394    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:06.720646 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.720394    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:06.720693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.720404    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:06.726085 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.726068    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 12:10:06.730676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.730659    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 12:10:06.731001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.730989    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hpk75\""
Sep 10 12:10:06.731230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.731214    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7mt6m\""
Sep 10 12:10:06.739605 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.739587    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:06.855521 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.855495    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x"]
Sep 10 12:10:06.858526 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.858340    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zlph5"]
Sep 10 12:10:06.865441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.865004    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl"]
Sep 10 12:10:06.866513 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:06.866479    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d29e9a0_2c23_42b5_80b1_2a961a66e48e.slice/crio-b68d3e541a46070be12fbaa0727c168d46d5f5000cd18e77c090efd040478ba2 WatchSource:0}: Error finding container b68d3e541a46070be12fbaa0727c168d46d5f5000cd18e77c090efd040478ba2: Status 404 returned error can't find the container with id b68d3e541a46070be12fbaa0727c168d46d5f5000cd18e77c090efd040478ba2
Sep 10 12:10:06.873663 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.873643    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-m646p"]
Sep 10 12:10:06.874437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.874362    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zlph5" event={"ID":"2d29e9a0-2c23-42b5-80b1-2a961a66e48e","Type":"ContainerStarted","Data":"b68d3e541a46070be12fbaa0727c168d46d5f5000cd18e77c090efd040478ba2"}
Sep 10 12:10:06.874762 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:06.874743    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15caee24_9bdb_490a_a632_ba9b4ecbede4.slice/crio-e9607fc1090b407a63a497fed36fb5dc1f590b39a6e6f0165920ab9a43191e49 WatchSource:0}: Error finding container e9607fc1090b407a63a497fed36fb5dc1f590b39a6e6f0165920ab9a43191e49: Status 404 returned error can't find the container with id e9607fc1090b407a63a497fed36fb5dc1f590b39a6e6f0165920ab9a43191e49
Sep 10 12:10:06.874908 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.874882    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-tgczq"]
Sep 10 12:10:06.875077 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:06.875041    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40af8ac6_a203_4ac5_8e54_64baac884dda.slice/crio-14d0afc13417f4ac88bf00ee35f8f466dcf1ffa7c2319a5b7787d63e0b11e6aa WatchSource:0}: Error finding container 14d0afc13417f4ac88bf00ee35f8f466dcf1ffa7c2319a5b7787d63e0b11e6aa: Status 404 returned error can't find the container with id 14d0afc13417f4ac88bf00ee35f8f466dcf1ffa7c2319a5b7787d63e0b11e6aa
Sep 10 12:10:06.875666 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.875630    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl" event={"ID":"90fb8de4-a0c3-4250-bca8-44c17293d423","Type":"ContainerStarted","Data":"e4b1c00a9d5ce587caaa37cdfff71cb406e724a84665c5cdff48ba5f3e6a843c"}
Sep 10 12:10:06.877210 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.877190    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x" event={"ID":"0f1a4986-32cd-45f9-86b6-3eb5410b7069","Type":"ContainerStarted","Data":"54c1b3600bc7f9f574b5454cc4e34839a3f619eda0a11c3e3dd7952c1aa77d02"}
Sep 10 12:10:06.878522 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.878503    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lrtv7"]
Sep 10 12:10:06.879563 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:06.879547    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv"]
Sep 10 12:10:06.880787 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:06.880764    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886da443_9220_47dc_9222_7b1aca3a6e78.slice/crio-badcb0a60380be86d69cd95464deba16cc03fa3aff0370b31b322355eb56b757 WatchSource:0}: Error finding container badcb0a60380be86d69cd95464deba16cc03fa3aff0370b31b322355eb56b757: Status 404 returned error can't find the container with id badcb0a60380be86d69cd95464deba16cc03fa3aff0370b31b322355eb56b757
Sep 10 12:10:06.881719 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:06.881701    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82423b91_84fc_44e5_b3a3_5f3c23ee4452.slice/crio-35e6e6e3b2bc0554be8e3b581383d70ad80c5a6e7d62615624e2e3237490e070 WatchSource:0}: Error finding container 35e6e6e3b2bc0554be8e3b581383d70ad80c5a6e7d62615624e2e3237490e070: Status 404 returned error can't find the container with id 35e6e6e3b2bc0554be8e3b581383d70ad80c5a6e7d62615624e2e3237490e070
Sep 10 12:10:07.016481 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.016452    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bn48x"]
Sep 10 12:10:07.052304 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:07.052278    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b09d5b_dc17_40b2_9f1c_1730ad7f88a5.slice/crio-b8bff40d13820918c97585034e2ccd8d06fbe55c896d9528d4423157dae8410c WatchSource:0}: Error finding container b8bff40d13820918c97585034e2ccd8d06fbe55c896d9528d4423157dae8410c: Status 404 returned error can't find the container with id b8bff40d13820918c97585034e2ccd8d06fbe55c896d9528d4423157dae8410c
Sep 10 12:10:07.266110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.266084    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:07.266214 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.266195    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:07.266214 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.266205    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:07.266280 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.266244    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.26623306 +0000 UTC m=+37.046790511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:07.367173 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.367148    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:07.367269 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.367195    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:07.367269 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.367221    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:07.367269 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.367247    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367288    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.367273472 +0000 UTC m=+37.147830923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367352    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367352    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367414    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls podName:32ad9dc7-8703-4e37-8708-b28b17491b29 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.367401179 +0000 UTC m=+37.147958636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-5t45d" (UID: "32ad9dc7-8703-4e37-8708-b28b17491b29") : secret "samples-operator-tls" not found
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367434    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.367420507 +0000 UTC m=+37.147977975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:07.367442 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367354    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:07.367636 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.367467    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.367459165 +0000 UTC m=+37.148016617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:07.468697 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.468668    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:07.468829 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.468735    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:07.468829 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.468811    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:07.468829 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.468819    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:07.468971 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.468872    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.468856045 +0000 UTC m=+37.249413499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:07.468971 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:07.468890    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:09.468881121 +0000 UTC m=+37.249438575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:07.885214 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.885132    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-m646p" event={"ID":"15caee24-9bdb-490a-a632-ba9b4ecbede4","Type":"ContainerStarted","Data":"e9607fc1090b407a63a497fed36fb5dc1f590b39a6e6f0165920ab9a43191e49"}
Sep 10 12:10:07.889004 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.888912    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bn48x" event={"ID":"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5","Type":"ContainerStarted","Data":"b8bff40d13820918c97585034e2ccd8d06fbe55c896d9528d4423157dae8410c"}
Sep 10 12:10:07.896419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.895808    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="ff8b524cc81c7af86e8a9bdd5c12efabd7bb6220d90bc1b5b5e092a9a01efb77" exitCode=0
Sep 10 12:10:07.896419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.895879    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"ff8b524cc81c7af86e8a9bdd5c12efabd7bb6220d90bc1b5b5e092a9a01efb77"}
Sep 10 12:10:07.902470 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.902421    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq" event={"ID":"40af8ac6-a203-4ac5-8e54-64baac884dda","Type":"ContainerStarted","Data":"14d0afc13417f4ac88bf00ee35f8f466dcf1ffa7c2319a5b7787d63e0b11e6aa"}
Sep 10 12:10:07.905426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.905402    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lrtv7" event={"ID":"82423b91-84fc-44e5-b3a3-5f3c23ee4452","Type":"ContainerStarted","Data":"35e6e6e3b2bc0554be8e3b581383d70ad80c5a6e7d62615624e2e3237490e070"}
Sep 10 12:10:07.907206 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:07.907184    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv" event={"ID":"886da443-9220-47dc-9222-7b1aca3a6e78","Type":"ContainerStarted","Data":"badcb0a60380be86d69cd95464deba16cc03fa3aff0370b31b322355eb56b757"}
Sep 10 12:10:08.915607 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:08.915572    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8011ac5-40aa-4a34-9730-b3ba5fdf0035" containerID="e35cf3236a7772c48047e66ccde8188e9aa9fec264f52a2772fe31c28943081c" exitCode=0
Sep 10 12:10:08.916252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:08.915625    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerDied","Data":"e35cf3236a7772c48047e66ccde8188e9aa9fec264f52a2772fe31c28943081c"}
Sep 10 12:10:09.287086 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.286723    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:09.287086 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.286886    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:09.287086 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.286901    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:09.287086 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.286955    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.28693796 +0000 UTC m=+41.067495427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.387442    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.387521    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.387578    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.387622    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387721    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.387703426 +0000 UTC m=+41.168260884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387805    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:09.387851 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387850    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.387839936 +0000 UTC m=+41.168397391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:09.388313 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387904    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 12:10:09.388313 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387934    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls podName:32ad9dc7-8703-4e37-8708-b28b17491b29 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.38792478 +0000 UTC m=+41.168482238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-5t45d" (UID: "32ad9dc7-8703-4e37-8708-b28b17491b29") : secret "samples-operator-tls" not found
Sep 10 12:10:09.388313 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.387998    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:09.388313 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.388048    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.388031494 +0000 UTC m=+41.168588945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:09.488948 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.488912    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:09.489104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:09.488988    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:09.489104 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.489071    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:09.489259 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.489135    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.489117607 +0000 UTC m=+41.269675062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:09.489259 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.489142    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:09.489259 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:09.489196    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:13.489181555 +0000 UTC m=+41.269739011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:13.324736 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.324692    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:13.325252 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.324876    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:13.325252 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.324900    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:13.325252 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.324962    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.324947158 +0000 UTC m=+49.105504629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:13.425974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.425942    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:13.426156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.425994    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:13.426156 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426083    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:13.426156 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426103    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.426086209 +0000 UTC m=+49.206643660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:13.426156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.426131    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.426165    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426195    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.426183939 +0000 UTC m=+49.206741390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426227    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426248    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426277    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.426265792 +0000 UTC m=+49.206823243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:13.426355 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.426307    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls podName:32ad9dc7-8703-4e37-8708-b28b17491b29 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.426289816 +0000 UTC m=+49.206847272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-5t45d" (UID: "32ad9dc7-8703-4e37-8708-b28b17491b29") : secret "samples-operator-tls" not found
Sep 10 12:10:13.526985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.526968    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:13.527103 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:13.527004    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:13.527103 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.527085    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:13.527103 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.527097    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:13.527199 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.527128    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.527115959 +0000 UTC m=+49.307673410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:13.527199 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:13.527141    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:21.527135543 +0000 UTC m=+49.307692994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:14.935168 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.934433    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl" event={"ID":"90fb8de4-a0c3-4250-bca8-44c17293d423","Type":"ContainerStarted","Data":"454b8c8f2df16c492d392f7e54a48515121aad0410ed6d12e7071dec7177edd0"}
Sep 10 12:10:14.937875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.937475    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-m646p" event={"ID":"15caee24-9bdb-490a-a632-ba9b4ecbede4","Type":"ContainerStarted","Data":"2e0dd8c27e90cf8bdb826ac75e447ac648a5b2854579decbcff063be61958177"}
Sep 10 12:10:14.941663 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.941048    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bn48x" event={"ID":"71b09d5b-dc17-40b2-9f1c-1730ad7f88a5","Type":"ContainerStarted","Data":"b67ff9899377049f4673325d07a5bf8ec7ec7815e21e80a573b4a998f6396749"}
Sep 10 12:10:14.941663 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.941438    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:14.945263 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.945240    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" event={"ID":"e8011ac5-40aa-4a34-9730-b3ba5fdf0035","Type":"ContainerStarted","Data":"c77539d7e5717a424bfe2a873f38984161e866b3624a0ad67df2ba83e1818a6c"}
Sep 10 12:10:14.947418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.947399    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/0.log"
Sep 10 12:10:14.947509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.947431    2551 generic.go:356] "Generic (PLEG): container finished" podID="2d29e9a0-2c23-42b5-80b1-2a961a66e48e" containerID="8678b605d08dbed5c295f27c927368064a87d200882d47052c2a749c11c59ef3" exitCode=255
Sep 10 12:10:14.947509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.947489    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zlph5" event={"ID":"2d29e9a0-2c23-42b5-80b1-2a961a66e48e","Type":"ContainerDied","Data":"8678b605d08dbed5c295f27c927368064a87d200882d47052c2a749c11c59ef3"}
Sep 10 12:10:14.947991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.947728    2551 scope.go:117] "RemoveContainer" containerID="8678b605d08dbed5c295f27c927368064a87d200882d47052c2a749c11c59ef3"
Sep 10 12:10:14.949692 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.949658    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq" event={"ID":"40af8ac6-a203-4ac5-8e54-64baac884dda","Type":"ContainerStarted","Data":"5c9cc418f131e37bad99a5a8e2457c79219cbaf593f56fa4637ee69ad8d954ea"}
Sep 10 12:10:14.956739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.956719    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lrtv7" event={"ID":"82423b91-84fc-44e5-b3a3-5f3c23ee4452","Type":"ContainerStarted","Data":"0e56bd87f151d2fddb27551f1353dbc4362b26da2648abb1dda7cfb93c8cde77"}
Sep 10 12:10:14.970146 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.970128    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x" event={"ID":"0f1a4986-32cd-45f9-86b6-3eb5410b7069","Type":"ContainerStarted","Data":"f87395f2f5d908a938fde2b424e10e299963bd7fa0642feacf407b3096ff7020"}
Sep 10 12:10:14.972481 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.972460    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv" event={"ID":"886da443-9220-47dc-9222-7b1aca3a6e78","Type":"ContainerStarted","Data":"ae8a41942ab90adc32e7100ea1f5295cc50771328e2eb821ec8adf563caabf81"}
Sep 10 12:10:14.980230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:14.980151    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl" podStartSLOduration=8.491211788 podStartE2EDuration="15.98013517s" totalImagesPullingTime="7.488923382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:14.957280365 +0000 UTC m=+42.737837855" watchObservedRunningTime="2026-09-10 12:10:14.98013517 +0000 UTC m=+42.760692643"
Sep 10 12:10:15.006940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.005549    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q6vsb" podStartSLOduration=13.992478452 podStartE2EDuration="43.005533409s" totalImagesPullingTime="29.013054957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.003738157 +0000 UTC m=+42.784295632" watchObservedRunningTime="2026-09-10 12:10:15.005533409 +0000 UTC m=+42.786090882"
Sep 10 12:10:15.026523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.026391    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lrtv7" podStartSLOduration=2.553953502 podStartE2EDuration="10.026361329s" totalImagesPullingTime="7.472407827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.02361242 +0000 UTC m=+42.804169893" watchObservedRunningTime="2026-09-10 12:10:15.026361329 +0000 UTC m=+42.806918802"
Sep 10 12:10:15.042023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.041979    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-m646p" podStartSLOduration=8.561163064 podStartE2EDuration="16.041967002s" totalImagesPullingTime="7.480803938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.040173247 +0000 UTC m=+42.820730719" watchObservedRunningTime="2026-09-10 12:10:15.041967002 +0000 UTC m=+42.822524475"
Sep 10 12:10:15.065536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.065492    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq" podStartSLOduration=8.58672207 podStartE2EDuration="16.065476306s" totalImagesPullingTime="7.478754236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.064553474 +0000 UTC m=+42.845110973" watchObservedRunningTime="2026-09-10 12:10:15.065476306 +0000 UTC m=+42.846033781"
Sep 10 12:10:15.086428 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.086364    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bn48x" podStartSLOduration=35.784891778 podStartE2EDuration="43.086346873s" totalImagesPullingTime="7.301455095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.084939429 +0000 UTC m=+42.865496902" watchObservedRunningTime="2026-09-10 12:10:15.086346873 +0000 UTC m=+42.866904348"
Sep 10 12:10:15.113527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.113486    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv" podStartSLOduration=8.260272553 podStartE2EDuration="16.113473578s" totalImagesPullingTime="7.853201025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.112123175 +0000 UTC m=+42.892680663" watchObservedRunningTime="2026-09-10 12:10:15.113473578 +0000 UTC m=+42.894031082"
Sep 10 12:10:15.147437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.147368    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ttt4x" podStartSLOduration=9.577360173 podStartE2EDuration="16.147349382s" totalImagesPullingTime="6.569989209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.146285129 +0000 UTC m=+42.926842628" watchObservedRunningTime="2026-09-10 12:10:15.147349382 +0000 UTC m=+42.927906857"
Sep 10 12:10:15.465314 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.465277    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jlkgc"]
Sep 10 12:10:15.485020 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.484990    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jlkgc"]
Sep 10 12:10:15.485186 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.485087    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.488622 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.488456    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 12:10:15.488622 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.488501    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 12:10:15.488798 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.488720    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwz2g\""
Sep 10 12:10:15.546865 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.546840    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3906a31-caa0-4235-acc1-dd77ad2624f0-data-volume\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.546985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.546896    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.547052 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.547007    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64t8b\" (UniqueName: \"kubernetes.io/projected/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-api-access-64t8b\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.547107 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.547072    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3906a31-caa0-4235-acc1-dd77ad2624f0-crio-socket\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.547158 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.547137    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648305    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64t8b\" (UniqueName: \"kubernetes.io/projected/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-api-access-64t8b\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648360    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3906a31-caa0-4235-acc1-dd77ad2624f0-crio-socket\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648461    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3906a31-caa0-4235-acc1-dd77ad2624f0-crio-socket\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648579    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648632    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3906a31-caa0-4235-acc1-dd77ad2624f0-data-volume\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.648684    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.648764 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:15.648750    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:15.648948 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:15.648780    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 12:10:15.648948 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:15.648808    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:16.148790412 +0000 UTC m=+43.929347870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:15.648948 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:15.648827    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:16.148817853 +0000 UTC m=+43.929375308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : configmap "kube-rbac-proxy" not found
Sep 10 12:10:15.649121 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.649102    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3906a31-caa0-4235-acc1-dd77ad2624f0-data-volume\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.665687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.665658    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64t8b\" (UniqueName: \"kubernetes.io/projected/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-api-access-64t8b\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:15.858166 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.858125    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:15.858313 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.858208    2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:15.977786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.977684    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:10:15.978227 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.978209    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/0.log"
Sep 10 12:10:15.978294 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.978246    2551 generic.go:356] "Generic (PLEG): container finished" podID="2d29e9a0-2c23-42b5-80b1-2a961a66e48e" containerID="3c7baaf0326e1d7b3e04d1c8e92d67d3922f9f26595b8b4a7873da0195e2838d" exitCode=255
Sep 10 12:10:15.978415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.978348    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zlph5" event={"ID":"2d29e9a0-2c23-42b5-80b1-2a961a66e48e","Type":"ContainerDied","Data":"3c7baaf0326e1d7b3e04d1c8e92d67d3922f9f26595b8b4a7873da0195e2838d"}
Sep 10 12:10:15.978595 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.978578    2551 scope.go:117] "RemoveContainer" containerID="8678b605d08dbed5c295f27c927368064a87d200882d47052c2a749c11c59ef3"
Sep 10 12:10:15.978655 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:15.978604    2551 scope.go:117] "RemoveContainer" containerID="3c7baaf0326e1d7b3e04d1c8e92d67d3922f9f26595b8b4a7873da0195e2838d"
Sep 10 12:10:15.978807 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:15.978782    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-zlph5_openshift-console-operator(2d29e9a0-2c23-42b5-80b1-2a961a66e48e)\"" pod="openshift-console-operator/console-operator-85b454874b-zlph5" podUID="2d29e9a0-2c23-42b5-80b1-2a961a66e48e"
Sep 10 12:10:16.158572 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.158536    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:16.158707 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:16.158675    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:16.158774 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:16.158717    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 12:10:16.158774 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.158674    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:16.158774 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:16.158739    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:17.158722173 +0000 UTC m=+44.939279627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:16.158774 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:16.158773    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:17.158754679 +0000 UTC m=+44.939312135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : configmap "kube-rbac-proxy" not found
Sep 10 12:10:16.440391 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.440356    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-569pc_955654cc-3b3c-4320-bb69-76ba876dc342/dns-node-resolver/0.log"
Sep 10 12:10:16.983054 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.983029    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:10:16.987577 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.987553    2551 scope.go:117] "RemoveContainer" containerID="3c7baaf0326e1d7b3e04d1c8e92d67d3922f9f26595b8b4a7873da0195e2838d"
Sep 10 12:10:16.987742 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:16.987721    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-zlph5_openshift-console-operator(2d29e9a0-2c23-42b5-80b1-2a961a66e48e)\"" pod="openshift-console-operator/console-operator-85b454874b-zlph5" podUID="2d29e9a0-2c23-42b5-80b1-2a961a66e48e"
Sep 10 12:10:16.991113 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:16.991085    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"]
Sep 10 12:10:17.024007 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.023985    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"
Sep 10 12:10:17.024414 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.024396    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"]
Sep 10 12:10:17.026525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.026504    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 12:10:17.026713 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.026695    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-j8k6f\""
Sep 10 12:10:17.026833 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.026819    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 12:10:17.066918 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.066891    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-929v4\" (UniqueName: \"kubernetes.io/projected/0ef0240a-a279-4c92-a8d3-e7ef83b5aae4-kube-api-access-929v4\") pod \"migrator-fdbb8fdff-rhrds\" (UID: \"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"
Sep 10 12:10:17.167569 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.167549    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:17.167664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.167606    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:17.167704 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:17.167674    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:17.167704 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.167688    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-929v4\" (UniqueName: \"kubernetes.io/projected/0ef0240a-a279-4c92-a8d3-e7ef83b5aae4-kube-api-access-929v4\") pod \"migrator-fdbb8fdff-rhrds\" (UID: \"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"
Sep 10 12:10:17.167764 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:17.167730    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:19.167716633 +0000 UTC m=+46.948274085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:17.167764 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:17.167745    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 12:10:17.167830 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:17.167792    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:19.167778363 +0000 UTC m=+46.948335825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : configmap "kube-rbac-proxy" not found
Sep 10 12:10:17.221360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.221338    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-929v4\" (UniqueName: \"kubernetes.io/projected/0ef0240a-a279-4c92-a8d3-e7ef83b5aae4-kube-api-access-929v4\") pod \"migrator-fdbb8fdff-rhrds\" (UID: \"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"
Sep 10 12:10:17.332906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.332888    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"
Sep 10 12:10:17.457015 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.456990    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9w79l_5f70ccae-062e-4a1d-8191-20e3b9e49c33/node-ca/0.log"
Sep 10 12:10:17.647818 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.647795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds"]
Sep 10 12:10:17.650611 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:17.650588    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef0240a_a279_4c92_a8d3_e7ef83b5aae4.slice/crio-a80a0c11ca6aabcfe712f1e93f9a901fc3ce089a9739e21c196e88abc7ac9b63 WatchSource:0}: Error finding container a80a0c11ca6aabcfe712f1e93f9a901fc3ce089a9739e21c196e88abc7ac9b63: Status 404 returned error can't find the container with id a80a0c11ca6aabcfe712f1e93f9a901fc3ce089a9739e21c196e88abc7ac9b63
Sep 10 12:10:17.990461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:17.990393    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds" event={"ID":"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4","Type":"ContainerStarted","Data":"a80a0c11ca6aabcfe712f1e93f9a901fc3ce089a9739e21c196e88abc7ac9b63"}
Sep 10 12:10:18.236878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.236855    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lrtv7_82423b91-84fc-44e5-b3a3-5f3c23ee4452/serve-healthcheck-canary/0.log"
Sep 10 12:10:18.608475 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.608445    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5x57q"]
Sep 10 12:10:18.626633 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.626607    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5x57q"]
Sep 10 12:10:18.626767 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.626720    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.629675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.629526    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 12:10:18.629675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.629644    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 12:10:18.630539 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.630433    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f27pd\""
Sep 10 12:10:18.630539 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.630503    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 12:10:18.630749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.630554    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 12:10:18.678981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.678959    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6613a564-9569-4d6a-8fc1-e176271b946a-signing-key\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.679073 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.678995    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6613a564-9569-4d6a-8fc1-e176271b946a-signing-cabundle\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.679150 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.679128    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjwp\" (UniqueName: \"kubernetes.io/projected/6613a564-9569-4d6a-8fc1-e176271b946a-kube-api-access-4qjwp\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.780355 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.780325    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjwp\" (UniqueName: \"kubernetes.io/projected/6613a564-9569-4d6a-8fc1-e176271b946a-kube-api-access-4qjwp\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.780614 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.780588    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6613a564-9569-4d6a-8fc1-e176271b946a-signing-key\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.780746 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.780629    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6613a564-9569-4d6a-8fc1-e176271b946a-signing-cabundle\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.781326 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.781306    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6613a564-9569-4d6a-8fc1-e176271b946a-signing-cabundle\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.783365 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.783345    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6613a564-9569-4d6a-8fc1-e176271b946a-signing-key\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.790742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.790722    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjwp\" (UniqueName: \"kubernetes.io/projected/6613a564-9569-4d6a-8fc1-e176271b946a-kube-api-access-4qjwp\") pod \"service-ca-75d99c484-5x57q\" (UID: \"6613a564-9569-4d6a-8fc1-e176271b946a\") " pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:18.938566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:18.938505    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-5x57q"
Sep 10 12:10:19.183980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.183943    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:19.184336 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:19.184062    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 12:10:19.184336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.184122    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:19.184336 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:19.184141    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:23.184123651 +0000 UTC m=+50.964681107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : configmap "kube-rbac-proxy" not found
Sep 10 12:10:19.184336 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:19.184194    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:19.184336 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:19.184235    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:23.184224626 +0000 UTC m=+50.964782077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:19.499662 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.499634    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5x57q"]
Sep 10 12:10:19.530064 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:19.530032    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6613a564_9569_4d6a_8fc1_e176271b946a.slice/crio-71324adcb63a5f048cdacc76d394c9f0e32fb21eda451113f27ecab827ac4826 WatchSource:0}: Error finding container 71324adcb63a5f048cdacc76d394c9f0e32fb21eda451113f27ecab827ac4826: Status 404 returned error can't find the container with id 71324adcb63a5f048cdacc76d394c9f0e32fb21eda451113f27ecab827ac4826
Sep 10 12:10:19.998120 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.998091    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds" event={"ID":"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4","Type":"ContainerStarted","Data":"a14c8d8b98d6d96d0ba2e555415b16dff21adb34abb80a8c794234df90f2347d"}
Sep 10 12:10:19.998255 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.998146    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds" event={"ID":"0ef0240a-a279-4c92-a8d3-e7ef83b5aae4","Type":"ContainerStarted","Data":"52d7787d7fd0a7ea2859bea69c55bde39140fd3395cf899a24fdfdc382606945"}
Sep 10 12:10:19.999513 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.999492    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-5x57q" event={"ID":"6613a564-9569-4d6a-8fc1-e176271b946a","Type":"ContainerStarted","Data":"9330e7d82575fe8010c2ec7e93432e189b70af081f272d1ed0c1c388bec4f804"}
Sep 10 12:10:19.999612 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:19.999530    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-5x57q" event={"ID":"6613a564-9569-4d6a-8fc1-e176271b946a","Type":"ContainerStarted","Data":"71324adcb63a5f048cdacc76d394c9f0e32fb21eda451113f27ecab827ac4826"}
Sep 10 12:10:20.020812 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:20.020769    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-rhrds" podStartSLOduration=2.116463471 podStartE2EDuration="4.020751968s" totalImagesPullingTime="1.904288497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:20.019938694 +0000 UTC m=+47.800496167" watchObservedRunningTime="2026-09-10 12:10:20.020751968 +0000 UTC m=+47.801309442"
Sep 10 12:10:20.040891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:20.040832    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-5x57q" podStartSLOduration=2.040820826 podStartE2EDuration="2.040820826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:20.040715019 +0000 UTC m=+47.821272491" watchObservedRunningTime="2026-09-10 12:10:20.040820826 +0000 UTC m=+47.821378299"
Sep 10 12:10:21.405368 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.405338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:21.405713 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.405470    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 12:10:21.405713 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.405490    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-679f9b85f8-v64kk: secret "image-registry-tls" not found
Sep 10 12:10:21.405713 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.405546    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls podName:ab51ca46-1fb7-44f1-8b8e-1bab82772093 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.405528536 +0000 UTC m=+65.186085994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls") pod "image-registry-679f9b85f8-v64kk" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093") : secret "image-registry-tls" not found
Sep 10 12:10:21.506497 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.506468    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:21.506661 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.506611    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 12:10:21.506661 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.506639    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:21.506780 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.506675    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.506657537 +0000 UTC m=+65.287215003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : secret "router-metrics-certs-default" not found
Sep 10 12:10:21.506780 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.506720    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle podName:b45f74b2-bcbd-4d26-881f-ff82ec49c1eb nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.506709403 +0000 UTC m=+65.287266865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle") pod "router-default-6b6767dd58-vrxv6" (UID: "b45f74b2-bcbd-4d26-881f-ff82ec49c1eb") : configmap references non-existent config key: service-ca.crt
Sep 10 12:10:21.506780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.506743    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:21.506931 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.506785    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:21.506931 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.506832    2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:21.506931 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.506871    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls podName:b0ec98c0-424b-4746-a9d9-ba63f1eb343b nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.506861438 +0000 UTC m=+65.287418893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-krbld" (UID: "b0ec98c0-424b-4746-a9d9-ba63f1eb343b") : secret "cluster-monitoring-operator-tls" not found
Sep 10 12:10:21.509129 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.509096    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32ad9dc7-8703-4e37-8708-b28b17491b29-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-5t45d\" (UID: \"32ad9dc7-8703-4e37-8708-b28b17491b29\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:21.607960 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.607937    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:21.608036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.607976    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:21.608100 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.608082    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 12:10:21.608143 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.608101    2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 12:10:21.608143 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.608141    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls podName:5208d4e0-3d95-49be-80f3-f3e681fc63b5 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.608126277 +0000 UTC m=+65.388683748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls") pod "dns-default-29d4k" (UID: "5208d4e0-3d95-49be-80f3-f3e681fc63b5") : secret "dns-default-metrics-tls" not found
Sep 10 12:10:21.608217 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:21.608159    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert podName:08296382-68e4-4fa4-837b-47b37de1c881 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:37.608150123 +0000 UTC m=+65.388707576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-b8kql" (UID: "08296382-68e4-4fa4-837b-47b37de1c881") : secret "networking-console-plugin-cert" not found
Sep 10 12:10:21.793900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:21.793853    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"
Sep 10 12:10:22.095026 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:22.094897    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d"]
Sep 10 12:10:23.011444 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:23.011412    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d" event={"ID":"32ad9dc7-8703-4e37-8708-b28b17491b29","Type":"ContainerStarted","Data":"d3567f8c4726b1e13bf371871725d0d1914b86825486fb113a25168b25700951"}
Sep 10 12:10:23.221885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:23.221851    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:23.222060 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:23.221994    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:23.222060 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:23.222014    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:23.222170 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:10:23.222060    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls podName:e3906a31-caa0-4235-acc1-dd77ad2624f0 nodeName:}" failed. No retries permitted until 2026-09-10 12:10:31.222042488 +0000 UTC m=+59.002599959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlkgc" (UID: "e3906a31-caa0-4235-acc1-dd77ad2624f0") : secret "insights-runtime-extractor-tls" not found
Sep 10 12:10:23.222570 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:23.222551    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3906a31-caa0-4235-acc1-dd77ad2624f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:24.017422 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:24.017389    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d" event={"ID":"32ad9dc7-8703-4e37-8708-b28b17491b29","Type":"ContainerStarted","Data":"347b2c6b304b11ad366c7b25f88c56e67267dbcf5ea32f54d70fe7580c5e2504"}
Sep 10 12:10:24.017807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:24.017451    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d" event={"ID":"32ad9dc7-8703-4e37-8708-b28b17491b29","Type":"ContainerStarted","Data":"e074f8fa8de6a23d8c2be3ec79b3d59a21b68b08d07bac1a90c8f85e06414032"}
Sep 10 12:10:24.038577 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:24.038529    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-5t45d" podStartSLOduration=23.311486721 podStartE2EDuration="25.038512825s" totalImagesPullingTime="1.727026104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:24.038412708 +0000 UTC m=+51.818970207" watchObservedRunningTime="2026-09-10 12:10:24.038512825 +0000 UTC m=+51.819070301"
Sep 10 12:10:25.858217 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:25.858187    2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:25.858636 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:25.858257    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:25.858636 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:25.858545    2551 scope.go:117] "RemoveContainer" containerID="3c7baaf0326e1d7b3e04d1c8e92d67d3922f9f26595b8b4a7873da0195e2838d"
Sep 10 12:10:26.025897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:26.025869    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:10:26.026047 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:26.025929    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zlph5" event={"ID":"2d29e9a0-2c23-42b5-80b1-2a961a66e48e","Type":"ContainerStarted","Data":"b07ba6d6365869d4c3f180e352f8911450c336602fd0cd95108fe776505cf8ef"}
Sep 10 12:10:26.026213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:26.026193    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:26.050029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:26.049981    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-zlph5" podStartSLOduration=19.56578835 podStartE2EDuration="27.049963668s" totalImagesPullingTime="7.484175318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:26.048944521 +0000 UTC m=+53.829501991" watchObservedRunningTime="2026-09-10 12:10:26.049963668 +0000 UTC m=+53.830521142"
Sep 10 12:10:26.333081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:26.333050    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-zlph5"
Sep 10 12:10:30.865821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:30.865795    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nj4hc"
Sep 10 12:10:31.286869 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:31.286805    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:31.288809 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:31.288790    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3906a31-caa0-4235-acc1-dd77ad2624f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlkgc\" (UID: \"e3906a31-caa0-4235-acc1-dd77ad2624f0\") " pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:31.397437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:31.397415    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jlkgc"
Sep 10 12:10:31.664610 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:31.664580    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jlkgc"]
Sep 10 12:10:31.667120 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:31.667091    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3906a31_caa0_4235_acc1_dd77ad2624f0.slice/crio-b7d4dcc4b99223dfac9e54b028b0ff53e0380aa049eca97280a55b8ec8f227da WatchSource:0}: Error finding container b7d4dcc4b99223dfac9e54b028b0ff53e0380aa049eca97280a55b8ec8f227da: Status 404 returned error can't find the container with id b7d4dcc4b99223dfac9e54b028b0ff53e0380aa049eca97280a55b8ec8f227da
Sep 10 12:10:32.046769 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:32.046701    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlkgc" event={"ID":"e3906a31-caa0-4235-acc1-dd77ad2624f0","Type":"ContainerStarted","Data":"052d14925aca1a46de811820a39f7085725df2b2ce067dd76218d48255e5fef4"}
Sep 10 12:10:32.046769 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:32.046754    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlkgc" event={"ID":"e3906a31-caa0-4235-acc1-dd77ad2624f0","Type":"ContainerStarted","Data":"b7d4dcc4b99223dfac9e54b028b0ff53e0380aa049eca97280a55b8ec8f227da"}
Sep 10 12:10:33.051316 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:33.051282    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlkgc" event={"ID":"e3906a31-caa0-4235-acc1-dd77ad2624f0","Type":"ContainerStarted","Data":"58334cccf0b4601be90c8362ce49692c5318b185facd8d21d1e40b68169c21d0"}
Sep 10 12:10:35.060005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:35.059970    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlkgc" event={"ID":"e3906a31-caa0-4235-acc1-dd77ad2624f0","Type":"ContainerStarted","Data":"3282e7fa52cddd90d3888ca32b09b2375e9a8ba8f6e5a1a1215386f80bc2bf93"}
Sep 10 12:10:35.079496 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:35.079452    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jlkgc" podStartSLOduration=17.815304616 podStartE2EDuration="20.079438667s" totalImagesPullingTime="2.264134051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:10:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:35.078346188 +0000 UTC m=+62.858903682" watchObservedRunningTime="2026-09-10 12:10:35.079438667 +0000 UTC m=+62.859996139"
Sep 10 12:10:37.430472 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.430441    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:37.432744 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.432722    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"image-registry-679f9b85f8-v64kk\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") " pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:37.531368 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.531341    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:37.531489 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.531414    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:37.531489 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.531463    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:37.531930 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.531905    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-service-ca-bundle\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:37.533495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.533470    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45f74b2-bcbd-4d26-881f-ff82ec49c1eb-metrics-certs\") pod \"router-default-6b6767dd58-vrxv6\" (UID: \"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb\") " pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:37.533570 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.533543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b0ec98c0-424b-4746-a9d9-ba63f1eb343b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-krbld\" (UID: \"b0ec98c0-424b-4746-a9d9-ba63f1eb343b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:37.631830 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.631809    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:37.631926 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.631842    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:37.633799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.633778    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5208d4e0-3d95-49be-80f3-f3e681fc63b5-metrics-tls\") pod \"dns-default-29d4k\" (UID: \"5208d4e0-3d95-49be-80f3-f3e681fc63b5\") " pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:37.633862 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.633843    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08296382-68e4-4fa4-837b-47b37de1c881-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b8kql\" (UID: \"08296382-68e4-4fa4-837b-47b37de1c881\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:37.643471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.643454    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vctr\""
Sep 10 12:10:37.651443 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.651424    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:37.721661 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.721639    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hrltp\""
Sep 10 12:10:37.729016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.729000    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:37.787367 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.787340    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rqlwd\""
Sep 10 12:10:37.794927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.794907    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"
Sep 10 12:10:37.820532 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.820509    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7jv8\""
Sep 10 12:10:37.828193 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.828009    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql"
Sep 10 12:10:37.843495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.842933    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gxdf\""
Sep 10 12:10:37.850362 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.850330    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:37.920516 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:37.920490    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:10:37.923816 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:37.923790    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab51ca46_1fb7_44f1_8b8e_1bab82772093.slice/crio-b6b050ac02e83081c69d404c3538de385bd5ea1f73b24e6b0ed7702eb5eda403 WatchSource:0}: Error finding container b6b050ac02e83081c69d404c3538de385bd5ea1f73b24e6b0ed7702eb5eda403: Status 404 returned error can't find the container with id b6b050ac02e83081c69d404c3538de385bd5ea1f73b24e6b0ed7702eb5eda403
Sep 10 12:10:38.005874 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.005850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b6767dd58-vrxv6"]
Sep 10 12:10:38.008105 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:38.008084    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45f74b2_bcbd_4d26_881f_ff82ec49c1eb.slice/crio-e57e5ce9b9acf9fcf467e657f43aec0a795dfbff0e39475e04689b896c545eb6 WatchSource:0}: Error finding container e57e5ce9b9acf9fcf467e657f43aec0a795dfbff0e39475e04689b896c545eb6: Status 404 returned error can't find the container with id e57e5ce9b9acf9fcf467e657f43aec0a795dfbff0e39475e04689b896c545eb6
Sep 10 12:10:38.063194 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.063109    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld"]
Sep 10 12:10:38.065642 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:38.065621    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ec98c0_424b_4746_a9d9_ba63f1eb343b.slice/crio-c92609c8fb4f6217263f9e1b2932a16ceed85b20d208c741d49a01d8ad238688 WatchSource:0}: Error finding container c92609c8fb4f6217263f9e1b2932a16ceed85b20d208c741d49a01d8ad238688: Status 404 returned error can't find the container with id c92609c8fb4f6217263f9e1b2932a16ceed85b20d208c741d49a01d8ad238688
Sep 10 12:10:38.070566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.070540    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b6767dd58-vrxv6" event={"ID":"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb","Type":"ContainerStarted","Data":"e57e5ce9b9acf9fcf467e657f43aec0a795dfbff0e39475e04689b896c545eb6"}
Sep 10 12:10:38.072195 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.072174    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" event={"ID":"ab51ca46-1fb7-44f1-8b8e-1bab82772093","Type":"ContainerStarted","Data":"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"}
Sep 10 12:10:38.072296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.072209    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" event={"ID":"ab51ca46-1fb7-44f1-8b8e-1bab82772093","Type":"ContainerStarted","Data":"b6b050ac02e83081c69d404c3538de385bd5ea1f73b24e6b0ed7702eb5eda403"}
Sep 10 12:10:38.072296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.072257    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:10:38.073610 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.073589    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld" event={"ID":"b0ec98c0-424b-4746-a9d9-ba63f1eb343b","Type":"ContainerStarted","Data":"c92609c8fb4f6217263f9e1b2932a16ceed85b20d208c741d49a01d8ad238688"}
Sep 10 12:10:38.096150 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.096103    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" podStartSLOduration=45.096087566 podStartE2EDuration="45.096087566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:38.095679023 +0000 UTC m=+65.876236522" watchObservedRunningTime="2026-09-10 12:10:38.096087566 +0000 UTC m=+65.876645038"
Sep 10 12:10:38.107931 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.107907    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b8kql"]
Sep 10 12:10:38.110310 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:38.110286    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08296382_68e4_4fa4_837b_47b37de1c881.slice/crio-deee4bfedff5fa0f012371d040a104731c2eec7dbed9347d5bdd6575900f8282 WatchSource:0}: Error finding container deee4bfedff5fa0f012371d040a104731c2eec7dbed9347d5bdd6575900f8282: Status 404 returned error can't find the container with id deee4bfedff5fa0f012371d040a104731c2eec7dbed9347d5bdd6575900f8282
Sep 10 12:10:38.126394 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.126315    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-29d4k"]
Sep 10 12:10:38.129590 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:38.129563    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5208d4e0_3d95_49be_80f3_f3e681fc63b5.slice/crio-1522c950a70b9c2b6d483c14c9ce936c1616003925fe1c37f5323a2ea1968439 WatchSource:0}: Error finding container 1522c950a70b9c2b6d483c14c9ce936c1616003925fe1c37f5323a2ea1968439: Status 404 returned error can't find the container with id 1522c950a70b9c2b6d483c14c9ce936c1616003925fe1c37f5323a2ea1968439
Sep 10 12:10:38.539406 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.539354    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:38.539406 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.539410    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:38.541983 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.541965    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 12:10:38.542064 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.541997    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 12:10:38.552184 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.552158    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e1d71-1163-422a-b8aa-5653e3678ac5-metrics-certs\") pod \"network-metrics-daemon-frgs7\" (UID: \"de4e1d71-1163-422a-b8aa-5653e3678ac5\") " pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:38.552345 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.552329    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d86fe67c-88f7-4a0f-b640-c1150019d3fc-original-pull-secret\") pod \"global-pull-secret-syncer-x9z4k\" (UID: \"d86fe67c-88f7-4a0f-b640-c1150019d3fc\") " pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:38.836320 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.836289    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x9z4k"
Sep 10 12:10:38.845859 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.845837    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7mt6m\""
Sep 10 12:10:38.854213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:38.854193    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frgs7"
Sep 10 12:10:39.079229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.079192    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql" event={"ID":"08296382-68e4-4fa4-837b-47b37de1c881","Type":"ContainerStarted","Data":"deee4bfedff5fa0f012371d040a104731c2eec7dbed9347d5bdd6575900f8282"}
Sep 10 12:10:39.081010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.080970    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b6767dd58-vrxv6" event={"ID":"b45f74b2-bcbd-4d26-881f-ff82ec49c1eb","Type":"ContainerStarted","Data":"d92c14f0fb7ef5cb0a8842a31ca980d83e3de0541a01f2457b061c7101c423f2"}
Sep 10 12:10:39.082546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.082520    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-29d4k" event={"ID":"5208d4e0-3d95-49be-80f3-f3e681fc63b5","Type":"ContainerStarted","Data":"1522c950a70b9c2b6d483c14c9ce936c1616003925fe1c37f5323a2ea1968439"}
Sep 10 12:10:39.140618 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.140438    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6b6767dd58-vrxv6" podStartSLOduration=40.140418356 podStartE2EDuration="40.140418356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:39.1271081 +0000 UTC m=+66.907665574" watchObservedRunningTime="2026-09-10 12:10:39.140418356 +0000 UTC m=+66.920975834"
Sep 10 12:10:39.142079 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.142018    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x9z4k"]
Sep 10 12:10:39.146245 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:39.146218    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86fe67c_88f7_4a0f_b640_c1150019d3fc.slice/crio-133e7c94d659967266983d095a79160cd6d66d0cda011ee2998cba3e48a15dcf WatchSource:0}: Error finding container 133e7c94d659967266983d095a79160cd6d66d0cda011ee2998cba3e48a15dcf: Status 404 returned error can't find the container with id 133e7c94d659967266983d095a79160cd6d66d0cda011ee2998cba3e48a15dcf
Sep 10 12:10:39.167273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.167213    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-frgs7"]
Sep 10 12:10:39.171099 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:39.171074    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4e1d71_1163_422a_b8aa_5653e3678ac5.slice/crio-213cc4dd2958b1903c50ef244cf77d7964293a99fdadeef91067e8f0c8ec2e5b WatchSource:0}: Error finding container 213cc4dd2958b1903c50ef244cf77d7964293a99fdadeef91067e8f0c8ec2e5b: Status 404 returned error can't find the container with id 213cc4dd2958b1903c50ef244cf77d7964293a99fdadeef91067e8f0c8ec2e5b
Sep 10 12:10:39.729643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.729578    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:39.732489 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:39.732452    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:40.088460 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:40.088407    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frgs7" event={"ID":"de4e1d71-1163-422a-b8aa-5653e3678ac5","Type":"ContainerStarted","Data":"213cc4dd2958b1903c50ef244cf77d7964293a99fdadeef91067e8f0c8ec2e5b"}
Sep 10 12:10:40.090070 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:40.090022    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x9z4k" event={"ID":"d86fe67c-88f7-4a0f-b640-c1150019d3fc","Type":"ContainerStarted","Data":"133e7c94d659967266983d095a79160cd6d66d0cda011ee2998cba3e48a15dcf"}
Sep 10 12:10:40.090414 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:40.090355    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:40.091787 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:40.091765    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b6767dd58-vrxv6"
Sep 10 12:10:41.097405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.097024    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-29d4k" event={"ID":"5208d4e0-3d95-49be-80f3-f3e681fc63b5","Type":"ContainerStarted","Data":"995a9985f6b41ad794b3b871ba02f951e9d1696f9692304395eb8a9366c93279"}
Sep 10 12:10:41.097405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.097099    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:41.097405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.097127    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-29d4k" event={"ID":"5208d4e0-3d95-49be-80f3-f3e681fc63b5","Type":"ContainerStarted","Data":"0685f46550f54c7e01893f1dfc90c6239e80c732cb8861ac2e12802eb2d5ed8e"}
Sep 10 12:10:41.099216 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.099184    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld" event={"ID":"b0ec98c0-424b-4746-a9d9-ba63f1eb343b","Type":"ContainerStarted","Data":"b15624c6559b329d43ab558be5b06aa9878532ebdbce3e5d227bfaccfce15489"}
Sep 10 12:10:41.101177 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.101128    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql" event={"ID":"08296382-68e4-4fa4-837b-47b37de1c881","Type":"ContainerStarted","Data":"f2bf5b52676444a364d8e0ac472a26583cec99e69ac8c6036debde7394c46e5c"}
Sep 10 12:10:41.124296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.124231    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-29d4k" podStartSLOduration=33.655794731 podStartE2EDuration="36.124215205s" totalImagesPullingTime="2.468420474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:41.123167396 +0000 UTC m=+68.903724889" watchObservedRunningTime="2026-09-10 12:10:41.124215205 +0000 UTC m=+68.904772678"
Sep 10 12:10:41.144004 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:41.143957    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-krbld" podStartSLOduration=39.61111639 podStartE2EDuration="42.143939578s" totalImagesPullingTime="2.532823188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:41.143215986 +0000 UTC m=+68.923773489" watchObservedRunningTime="2026-09-10 12:10:41.143939578 +0000 UTC m=+68.924497053"
Sep 10 12:10:42.864588 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.864529    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-b8kql" podStartSLOduration=41.38204928 podStartE2EDuration="43.864512252s" totalImagesPullingTime="2.482462972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:41.167149256 +0000 UTC m=+68.947706733" watchObservedRunningTime="2026-09-10 12:10:42.864512252 +0000 UTC m=+70.645069724"
Sep 10 12:10:42.866617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.865909    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"]
Sep 10 12:10:42.877856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.877832    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"]
Sep 10 12:10:42.877982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.877955    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:42.885057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.884833    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 12:10:42.885057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.884855    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 12:10:42.885057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.884996    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4f5t5\""
Sep 10 12:10:42.885057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.885025    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 12:10:42.885457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.885442    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 12:10:42.885980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.885959    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:42.889403 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.889352    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 12:10:42.889969 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.889946    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"]
Sep 10 12:10:42.897871 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.897607    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.900359 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.900315    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"]
Sep 10 12:10:42.900877 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.900859    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 12:10:42.901038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.901023    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 12:10:42.905881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.905789    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 12:10:42.906247 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.906013    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 12:10:42.927423 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.926827    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"]
Sep 10 12:10:42.938334 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.938305    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"]
Sep 10 12:10:42.976180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976147    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976312 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976188    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0820755e-f6d2-4023-86c2-bd4a530c91e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:42.976312 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976247    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjt7m\" (UniqueName: \"kubernetes.io/projected/0820755e-f6d2-4023-86c2-bd4a530c91e0-kube-api-access-vjt7m\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976326    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976345    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976370    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0820755e-f6d2-4023-86c2-bd4a530c91e0-tmp\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976407    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a421e6-2606-4785-a0e1-451f43275e23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976432    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/13b1f1fa-247b-4123-97ba-6f39c278c8f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976447    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976463    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zfzj\" (UniqueName: \"kubernetes.io/projected/a6a421e6-2606-4785-a0e1-451f43275e23-kube-api-access-6zfzj\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:42.976797 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.976498    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wf6\" (UniqueName: \"kubernetes.io/projected/13b1f1fa-247b-4123-97ba-6f39c278c8f4-kube-api-access-h9wf6\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:42.986332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:42.986308    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"]
Sep 10 12:10:43.002441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.002418    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-sg47x"]
Sep 10 12:10:43.002576 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.002557    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:43.005416 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.005392    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vx97d\""
Sep 10 12:10:43.005564 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.005529    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 12:10:43.017606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.017555    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:10:43.018405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.018028    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:10:43.021336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.021317    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 12:10:43.021606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.021588    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nqx4g\""
Sep 10 12:10:43.021700 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.021683    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 12:10:43.033558 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.033536    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"]
Sep 10 12:10:43.034085 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.033578    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-sg47x"]
Sep 10 12:10:43.034085 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.033669    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.035609 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.035587    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:10:43.038022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.037794    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 12:10:43.038022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.037904    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 12:10:43.038022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.037942    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 12:10:43.039451 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.039254    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 12:10:43.040090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.039594    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 12:10:43.040090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.039875    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4h9f5\""
Sep 10 12:10:43.077140 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077108    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.077348 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077157    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjt7m\" (UniqueName: \"kubernetes.io/projected/0820755e-f6d2-4023-86c2-bd4a530c91e0-kube-api-access-vjt7m\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.077348 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077221    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.077348 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077277    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.077348 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077301    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077350    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077400    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0820755e-f6d2-4023-86c2-bd4a530c91e0-tmp\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077447    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a421e6-2606-4785-a0e1-451f43275e23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6j5d\" (UniqueName: \"kubernetes.io/projected/ff2fb597-34ab-4a41-914a-1c6262f54efb-kube-api-access-s6j5d\") pod \"downloads-85d98c998f-sg47x\" (UID: \"ff2fb597-34ab-4a41-914a-1c6262f54efb\") " pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077507    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/13b1f1fa-247b-4123-97ba-6f39c278c8f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077532    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caed5e87-fec1-4182-b625-2ea886deba82-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-72glj\" (UID: \"caed5e87-fec1-4182-b625-2ea886deba82\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:43.077591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077585    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077618    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zfzj\" (UniqueName: \"kubernetes.io/projected/a6a421e6-2606-4785-a0e1-451f43275e23-kube-api-access-6zfzj\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077653    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wf6\" (UniqueName: \"kubernetes.io/projected/13b1f1fa-247b-4123-97ba-6f39c278c8f4-kube-api-access-h9wf6\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077687    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7jzh\" (UniqueName: \"kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077713    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077743    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077770    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0820755e-f6d2-4023-86c2-bd4a530c91e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077790    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.078273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.077791    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0820755e-f6d2-4023-86c2-bd4a530c91e0-tmp\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.078692 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.078590    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a421e6-2606-4785-a0e1-451f43275e23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.080711 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.080669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/13b1f1fa-247b-4123-97ba-6f39c278c8f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:43.081007 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.080983    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.081007 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.081001    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0820755e-f6d2-4023-86c2-bd4a530c91e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.081007 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.081005    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-ca\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.081218 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.081196    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.082106 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.081999    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a6a421e6-2606-4785-a0e1-451f43275e23-hub\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.093587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.093541    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjt7m\" (UniqueName: \"kubernetes.io/projected/0820755e-f6d2-4023-86c2-bd4a530c91e0-kube-api-access-vjt7m\") pod \"klusterlet-addon-workmgr-55c8dd967c-4zwwd\" (UID: \"0820755e-f6d2-4023-86c2-bd4a530c91e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.093587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.093556    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wf6\" (UniqueName: \"kubernetes.io/projected/13b1f1fa-247b-4123-97ba-6f39c278c8f4-kube-api-access-h9wf6\") pod \"managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg\" (UID: \"13b1f1fa-247b-4123-97ba-6f39c278c8f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:43.094365 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.094341    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zfzj\" (UniqueName: \"kubernetes.io/projected/a6a421e6-2606-4785-a0e1-451f43275e23-kube-api-access-6zfzj\") pod \"cluster-proxy-proxy-agent-799876fd59-9qj8k\" (UID: \"a6a421e6-2606-4785-a0e1-451f43275e23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.178457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178392    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.178457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178445    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178485    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6j5d\" (UniqueName: \"kubernetes.io/projected/ff2fb597-34ab-4a41-914a-1c6262f54efb-kube-api-access-s6j5d\") pod \"downloads-85d98c998f-sg47x\" (UID: \"ff2fb597-34ab-4a41-914a-1c6262f54efb\") " pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178505    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caed5e87-fec1-4182-b625-2ea886deba82-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-72glj\" (UID: \"caed5e87-fec1-4182-b625-2ea886deba82\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178534    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7jzh\" (UniqueName: \"kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178565    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.178643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.178615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.179050 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.179024    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.179444 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.179424    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.179552 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.179532    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.180909 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.180829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.180990 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.180923    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.180990 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.180961    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caed5e87-fec1-4182-b625-2ea886deba82-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-72glj\" (UID: \"caed5e87-fec1-4182-b625-2ea886deba82\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:43.190621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.190598    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7jzh\" (UniqueName: \"kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh\") pod \"console-6f667dcc4-2cxtk\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") " pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:43.197927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.197910    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"
Sep 10 12:10:43.198573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.198555    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6j5d\" (UniqueName: \"kubernetes.io/projected/ff2fb597-34ab-4a41-914a-1c6262f54efb-kube-api-access-s6j5d\") pod \"downloads-85d98c998f-sg47x\" (UID: \"ff2fb597-34ab-4a41-914a-1c6262f54efb\") " pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:10:43.210350 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.210332    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:43.216514 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.216498    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"
Sep 10 12:10:43.323348 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.323319    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:43.332475 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.332449    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:10:43.345853 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:43.345814    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:10:44.180064 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.180038    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"]
Sep 10 12:10:44.182695 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:44.182672    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0820755e_f6d2_4023_86c2_bd4a530c91e0.slice/crio-3dca8880387fd10250afee5265073838678dad09b322c3947349810c73d332ff WatchSource:0}: Error finding container 3dca8880387fd10250afee5265073838678dad09b322c3947349810c73d332ff: Status 404 returned error can't find the container with id 3dca8880387fd10250afee5265073838678dad09b322c3947349810c73d332ff
Sep 10 12:10:44.198066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.197969    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg"]
Sep 10 12:10:44.199446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.199339    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:10:44.202105 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:44.202029    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184f81ce_57ce_447f_bbe0_b72179af0e46.slice/crio-ef2c4aa724f42d18d77b516afacaed18ca03a7784593028255cf148a664abfa9 WatchSource:0}: Error finding container ef2c4aa724f42d18d77b516afacaed18ca03a7784593028255cf148a664abfa9: Status 404 returned error can't find the container with id ef2c4aa724f42d18d77b516afacaed18ca03a7784593028255cf148a664abfa9
Sep 10 12:10:44.203265 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:44.203224    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13b1f1fa_247b_4123_97ba_6f39c278c8f4.slice/crio-725ab00dd78ae352a96845a649a5281c077ab8a6d4d7ac31fab0554a365f1435 WatchSource:0}: Error finding container 725ab00dd78ae352a96845a649a5281c077ab8a6d4d7ac31fab0554a365f1435: Status 404 returned error can't find the container with id 725ab00dd78ae352a96845a649a5281c077ab8a6d4d7ac31fab0554a365f1435
Sep 10 12:10:44.269405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.269345    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k"]
Sep 10 12:10:44.271165 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.271136    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"]
Sep 10 12:10:44.272765 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:44.272741    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-sg47x"]
Sep 10 12:10:44.273966 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:44.273886    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaed5e87_fec1_4182_b625_2ea886deba82.slice/crio-d33236d9fcb9536f404f49b52393184191b415017243f4f3119ebdc230f6e9db WatchSource:0}: Error finding container d33236d9fcb9536f404f49b52393184191b415017243f4f3119ebdc230f6e9db: Status 404 returned error can't find the container with id d33236d9fcb9536f404f49b52393184191b415017243f4f3119ebdc230f6e9db
Sep 10 12:10:44.274177 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:10:44.274145    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff2fb597_34ab_4a41_914a_1c6262f54efb.slice/crio-b1ae84be86e1500ddf622980c8c6130bba0b063d387840c6863438a2d2c1d54a WatchSource:0}: Error finding container b1ae84be86e1500ddf622980c8c6130bba0b063d387840c6863438a2d2c1d54a: Status 404 returned error can't find the container with id b1ae84be86e1500ddf622980c8c6130bba0b063d387840c6863438a2d2c1d54a
Sep 10 12:10:45.123626 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.123581    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k" event={"ID":"a6a421e6-2606-4785-a0e1-451f43275e23","Type":"ContainerStarted","Data":"9d3f0705d3a534c680bf2276334b27d096e51f9098199d4219d8dfbd62ba5792"}
Sep 10 12:10:45.125674 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.125548    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-sg47x" event={"ID":"ff2fb597-34ab-4a41-914a-1c6262f54efb","Type":"ContainerStarted","Data":"b1ae84be86e1500ddf622980c8c6130bba0b063d387840c6863438a2d2c1d54a"}
Sep 10 12:10:45.128473 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.128417    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg" event={"ID":"13b1f1fa-247b-4123-97ba-6f39c278c8f4","Type":"ContainerStarted","Data":"725ab00dd78ae352a96845a649a5281c077ab8a6d4d7ac31fab0554a365f1435"}
Sep 10 12:10:45.132767 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.131807    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frgs7" event={"ID":"de4e1d71-1163-422a-b8aa-5653e3678ac5","Type":"ContainerStarted","Data":"c545d0e1b273564363c4106bc946300487ba4dcc2a25db3e92935ae9d72668bc"}
Sep 10 12:10:45.132767 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.131862    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frgs7" event={"ID":"de4e1d71-1163-422a-b8aa-5653e3678ac5","Type":"ContainerStarted","Data":"7bad6b0931f77dce59b23b6b0e36ee83ea791a5eb612ef7032cbb8b3ecbdcdf3"}
Sep 10 12:10:45.134354 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.134294    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f667dcc4-2cxtk" event={"ID":"184f81ce-57ce-447f-bbe0-b72179af0e46","Type":"ContainerStarted","Data":"ef2c4aa724f42d18d77b516afacaed18ca03a7784593028255cf148a664abfa9"}
Sep 10 12:10:45.137450 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.136926    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x9z4k" event={"ID":"d86fe67c-88f7-4a0f-b640-c1150019d3fc","Type":"ContainerStarted","Data":"7140d727d1892ae64641a7b7f4dd3057bfe4ed06aa0ebf4f19897bdb76b0912a"}
Sep 10 12:10:45.140672 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.140598    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj" event={"ID":"caed5e87-fec1-4182-b625-2ea886deba82","Type":"ContainerStarted","Data":"d33236d9fcb9536f404f49b52393184191b415017243f4f3119ebdc230f6e9db"}
Sep 10 12:10:45.146538 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.146496    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd" event={"ID":"0820755e-f6d2-4023-86c2-bd4a530c91e0","Type":"ContainerStarted","Data":"3dca8880387fd10250afee5265073838678dad09b322c3947349810c73d332ff"}
Sep 10 12:10:45.151639 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.151593    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-frgs7" podStartSLOduration=68.416690256 podStartE2EDuration="1m13.151576822s" totalImagesPullingTime="4.734886566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:45.150280511 +0000 UTC m=+72.930838023" watchObservedRunningTime="2026-09-10 12:10:45.151576822 +0000 UTC m=+72.932134299"
Sep 10 12:10:45.168579 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:45.168447    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x9z4k" podStartSLOduration=67.452879123 podStartE2EDuration="1m12.168384692s" totalImagesPullingTime="4.715505569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:45.166759274 +0000 UTC m=+72.947316771" watchObservedRunningTime="2026-09-10 12:10:45.168384692 +0000 UTC m=+72.948942165"
Sep 10 12:10:46.985730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:46.985695    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bn48x"
Sep 10 12:10:51.108011 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:51.107979    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-29d4k"
Sep 10 12:10:54.199018 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.198979    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg" event={"ID":"13b1f1fa-247b-4123-97ba-6f39c278c8f4","Type":"ContainerStarted","Data":"af1bde9cd7e03554048c634b2efaa42bf46f05e1a96de22b9ed0cd37a9a741f3"}
Sep 10 12:10:54.200965 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.200909    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f667dcc4-2cxtk" event={"ID":"184f81ce-57ce-447f-bbe0-b72179af0e46","Type":"ContainerStarted","Data":"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"}
Sep 10 12:10:54.202750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.202722    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj" event={"ID":"caed5e87-fec1-4182-b625-2ea886deba82","Type":"ContainerStarted","Data":"c0d6a08988aa548bcebd3a2787e30cf8f0f813e01b235e55a995a9823916532b"}
Sep 10 12:10:54.202901 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.202878    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:54.204820 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.204558    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd" event={"ID":"0820755e-f6d2-4023-86c2-bd4a530c91e0","Type":"ContainerStarted","Data":"f26b704c89f3dba357fd8f0413a669a75ca11ed977e0f9e2699bba3012b1d7f9"}
Sep 10 12:10:54.204820 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.204803    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:54.206541 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.206505    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd"
Sep 10 12:10:54.208602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.208439    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k" event={"ID":"a6a421e6-2606-4785-a0e1-451f43275e23","Type":"ContainerStarted","Data":"d5f7261e9b6ebcc02732400fe7b03d0bf4ca4a5dd8d295ec61f815c747b21b64"}
Sep 10 12:10:54.208602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.208474    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k" event={"ID":"a6a421e6-2606-4785-a0e1-451f43275e23","Type":"ContainerStarted","Data":"fee34751aa55b6acb0b1300424c748e3af38416e910c0e7bfd7ce9ed1827be0f"}
Sep 10 12:10:54.208602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.208493    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k" event={"ID":"a6a421e6-2606-4785-a0e1-451f43275e23","Type":"ContainerStarted","Data":"098c1a67fdabe69b5bc14b7c329199dc84f198b675cadcbba3f7bb28bcdf9f8f"}
Sep 10 12:10:54.209103 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.209085    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj"
Sep 10 12:10:54.219622 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.219576    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcf98c9db-l5wlg" podStartSLOduration=2.945237245 podStartE2EDuration="12.219560007s" totalImagesPullingTime="9.274322762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:54.216802972 +0000 UTC m=+81.997360445" watchObservedRunningTime="2026-09-10 12:10:54.219560007 +0000 UTC m=+82.000117480"
Sep 10 12:10:54.245364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.245316    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-72glj" podStartSLOduration=3.037346154 podStartE2EDuration="12.245300232s" totalImagesPullingTime="9.207954078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:54.243950729 +0000 UTC m=+82.024508226" watchObservedRunningTime="2026-09-10 12:10:54.245300232 +0000 UTC m=+82.025857703"
Sep 10 12:10:54.265048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.265010    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c8dd967c-4zwwd" podStartSLOduration=2.955809467 podStartE2EDuration="12.264996855s" totalImagesPullingTime="9.309187388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:54.2638535 +0000 UTC m=+82.044410973" watchObservedRunningTime="2026-09-10 12:10:54.264996855 +0000 UTC m=+82.045554327"
Sep 10 12:10:54.287020 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.286974    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f667dcc4-2cxtk" podStartSLOduration=3.004634963 podStartE2EDuration="12.286958159s" totalImagesPullingTime="9.282323196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:54.286223386 +0000 UTC m=+82.066780885" watchObservedRunningTime="2026-09-10 12:10:54.286958159 +0000 UTC m=+82.067515632"
Sep 10 12:10:54.311763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:54.311714    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-799876fd59-9qj8k" podStartSLOduration=3.081505712 podStartE2EDuration="12.311696643s" totalImagesPullingTime="9.230190931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:54.309632911 +0000 UTC m=+82.090190758" watchObservedRunningTime="2026-09-10 12:10:54.311696643 +0000 UTC m=+82.092254118"
Sep 10 12:10:56.746363 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:56.745977    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:10:57.657048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:57.657003    2551 patch_prober.go:28] interesting pod/image-registry-679f9b85f8-v64kk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 12:10:57.657230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:57.657066    2551 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 12:10:59.087587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:10:59.087554    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:11:00.668006 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.667894    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s6djt"]
Sep 10 12:11:00.727946 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.727612    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.731015 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.730605    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 12:11:00.731015 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.730652    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 12:11:00.731015 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.730883    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6pwvt\""
Sep 10 12:11:00.733495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.731548    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 12:11:00.733495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.731774    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 12:11:00.836898 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.836869    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.836898 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.836911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-textfile\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.836940    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-metrics-client-ca\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.836972    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fxdq\" (UniqueName: \"kubernetes.io/projected/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-kube-api-access-9fxdq\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.837015    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.837042    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-sys\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.837060    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-wtmp\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.837110    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-root\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.837421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.837137    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938799    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-sys\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938827    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-wtmp\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938877    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-root\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938904    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:11:00.938932    2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938957    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.938987    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-textfile\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:11:00.939007    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls podName:8bff6cd3-776f-4b23-8e91-e8ccdf50cd43 nodeName:}" failed. No retries permitted until 2026-09-10 12:11:01.438985722 +0000 UTC m=+89.219543195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls") pod "node-exporter-s6djt" (UID: "8bff6cd3-776f-4b23-8e91-e8ccdf50cd43") : secret "node-exporter-tls" not found
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939048    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-metrics-client-ca\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939104 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939088    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fxdq\" (UniqueName: \"kubernetes.io/projected/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-kube-api-access-9fxdq\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939285    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-textfile\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939353    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-sys\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939519    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-wtmp\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.939741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939566    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-root\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.940008 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.939935    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-metrics-client-ca\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.940106 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.940030    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.943602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.943560    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:00.954279 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:00.954255    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fxdq\" (UniqueName: \"kubernetes.io/projected/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-kube-api-access-9fxdq\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:01.442908 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:01.442879    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:01.445403 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:01.445361    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8bff6cd3-776f-4b23-8e91-e8ccdf50cd43-node-exporter-tls\") pod \"node-exporter-s6djt\" (UID: \"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43\") " pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:01.642620 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:01.642591    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6djt"
Sep 10 12:11:03.346783 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:03.346745    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:11:03.862239 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:11:03.862204    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bff6cd3_776f_4b23_8e91_e8ccdf50cd43.slice/crio-fd2f6f8fd49234eac26ae1a68261336b65f97af26cde4fc5ab5ae3b37fbd393b WatchSource:0}: Error finding container fd2f6f8fd49234eac26ae1a68261336b65f97af26cde4fc5ab5ae3b37fbd393b: Status 404 returned error can't find the container with id fd2f6f8fd49234eac26ae1a68261336b65f97af26cde4fc5ab5ae3b37fbd393b
Sep 10 12:11:04.253422 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:04.253395    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6djt" event={"ID":"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43","Type":"ContainerStarted","Data":"fd2f6f8fd49234eac26ae1a68261336b65f97af26cde4fc5ab5ae3b37fbd393b"}
Sep 10 12:11:05.260109 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.260074    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-sg47x" event={"ID":"ff2fb597-34ab-4a41-914a-1c6262f54efb","Type":"ContainerStarted","Data":"77db2b8eba7cd40afbd35bcf76f3609395a78964fd41be969ac6bf52feecaad2"}
Sep 10 12:11:05.260565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.260302    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:11:05.275786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.275759    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-sg47x"
Sep 10 12:11:05.280514 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.280461    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-sg47x" podStartSLOduration=3.315638574 podStartE2EDuration="23.280442579s" totalImagesPullingTime="19.964804005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:05.27888373 +0000 UTC m=+93.059441222" watchObservedRunningTime="2026-09-10 12:11:05.280442579 +0000 UTC m=+93.061000053"
Sep 10 12:11:05.431775 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.431739    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:05.450721 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.449956    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.454562 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.453062    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:05.461050 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.460850    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 12:11:05.566633 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.566568    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580675    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580750    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkx2v\" (UniqueName: \"kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580780    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580804    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580838    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.581023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.580932    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.681865 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681837    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681874    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681906    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681936    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkx2v\" (UniqueName: \"kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681956    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.681970    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.682003    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.682594 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.682566    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.683129 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.683101    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.683592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.683574    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.683998 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.683953    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.685875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.685726    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.685875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.685832    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.694043 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.694023    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkx2v\" (UniqueName: \"kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v\") pod \"console-598cd55b69-gmcwr\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") " pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:05.766970 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:05.766934    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:06.070956 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.070924    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:06.073693 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:11:06.073664    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5bb786_eac8_4e21_b9dd_bfd1a21f0426.slice/crio-70d367f98c690e07b2215014bdcf97ca4a50107249802d5aa8fe3284004d6316 WatchSource:0}: Error finding container 70d367f98c690e07b2215014bdcf97ca4a50107249802d5aa8fe3284004d6316: Status 404 returned error can't find the container with id 70d367f98c690e07b2215014bdcf97ca4a50107249802d5aa8fe3284004d6316
Sep 10 12:11:06.267957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.267924    2551 generic.go:356] "Generic (PLEG): container finished" podID="8bff6cd3-776f-4b23-8e91-e8ccdf50cd43" containerID="62356a7a8160f52b118c0762f2c62fe36bead12b995e1a263a2f576a68effab6" exitCode=0
Sep 10 12:11:06.268441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.268088    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6djt" event={"ID":"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43","Type":"ContainerDied","Data":"62356a7a8160f52b118c0762f2c62fe36bead12b995e1a263a2f576a68effab6"}
Sep 10 12:11:06.270702 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.270613    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598cd55b69-gmcwr" event={"ID":"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426","Type":"ContainerStarted","Data":"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"}
Sep 10 12:11:06.270702 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.270676    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598cd55b69-gmcwr" event={"ID":"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426","Type":"ContainerStarted","Data":"70d367f98c690e07b2215014bdcf97ca4a50107249802d5aa8fe3284004d6316"}
Sep 10 12:11:06.309176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:06.309133    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-598cd55b69-gmcwr" podStartSLOduration=1.309116592 podStartE2EDuration="1.309116592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:06.308035591 +0000 UTC m=+94.088593093" watchObservedRunningTime="2026-09-10 12:11:06.309116592 +0000 UTC m=+94.089674065"
Sep 10 12:11:07.284712 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:07.284663    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6djt" event={"ID":"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43","Type":"ContainerStarted","Data":"c1e07ee32c992ff856c89e36cf22a976f7bd9a8508e0ad5faf40bd906c49e24d"}
Sep 10 12:11:07.285192 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:07.284773    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6djt" event={"ID":"8bff6cd3-776f-4b23-8e91-e8ccdf50cd43","Type":"ContainerStarted","Data":"8f0e40e349247c3b8bb2dc40311c4cfe1e0efbbdaf3f2851c732dedd690300c1"}
Sep 10 12:11:07.304733 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:07.304678    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s6djt" podStartSLOduration=5.548211868 podStartE2EDuration="7.304662752s" totalImagesPullingTime="1.667532191s" totalInitContainerRuntime="88.918693ms" isStatefulPod=true podCreationTimestamp="2026-09-10 12:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:07.303815916 +0000 UTC m=+95.084373412" watchObservedRunningTime="2026-09-10 12:11:07.304662752 +0000 UTC m=+95.085220235"
Sep 10 12:11:09.782129 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.782091    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:09.816807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.816774    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:11:09.872359 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.872323    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:11:09.872531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.872479    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923330 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923336    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923369    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923416    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qqs\" (UniqueName: \"kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923537 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923507    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923554    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:09.923714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:09.923584    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024060 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024013    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024076    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024103    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024139    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024167    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024202    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024225    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qqs\" (UniqueName: \"kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.024984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.024934    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.025149 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.025075    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.025225 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.025195    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.025792 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.025770    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.026964 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.026940    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.027154 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.027133    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.032347 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.032298    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qqs\" (UniqueName: \"kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs\") pod \"console-765f5b8846-wfpnc\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") " pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.183709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.183672    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:10.498244 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:10.498210    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:11:10.501252 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:11:10.501214    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37dfeea0_670a_493f_b04f_1c5cca100296.slice/crio-525a8243c873a404aed17dfe8ff353bc0c0782b9e4dbf72a58ad4dcacba0aca9 WatchSource:0}: Error finding container 525a8243c873a404aed17dfe8ff353bc0c0782b9e4dbf72a58ad4dcacba0aca9: Status 404 returned error can't find the container with id 525a8243c873a404aed17dfe8ff353bc0c0782b9e4dbf72a58ad4dcacba0aca9
Sep 10 12:11:11.312032 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:11.311995    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765f5b8846-wfpnc" event={"ID":"37dfeea0-670a-493f-b04f-1c5cca100296","Type":"ContainerStarted","Data":"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"}
Sep 10 12:11:11.312503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:11.312051    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765f5b8846-wfpnc" event={"ID":"37dfeea0-670a-493f-b04f-1c5cca100296","Type":"ContainerStarted","Data":"525a8243c873a404aed17dfe8ff353bc0c0782b9e4dbf72a58ad4dcacba0aca9"}
Sep 10 12:11:11.334093 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:11.334045    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-765f5b8846-wfpnc" podStartSLOduration=2.33403046 podStartE2EDuration="2.33403046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:11.332070876 +0000 UTC m=+99.112628368" watchObservedRunningTime="2026-09-10 12:11:11.33403046 +0000 UTC m=+99.114587933"
Sep 10 12:11:15.768423 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:15.768360    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:20.183933 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:20.183893    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:20.184423 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:20.183963    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:20.188439 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:20.188419    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:20.356965 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:20.356941    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:11:21.768492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:21.768429    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f667dcc4-2cxtk" podUID="184f81ce-57ce-447f-bbe0-b72179af0e46" containerName="console" containerID="cri-o://d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57" gracePeriod=15
Sep 10 12:11:22.037586 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.037528    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:11:22.126761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126733    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.126902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126778    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.126902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126803    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7jzh\" (UniqueName: \"kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.126902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126836    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.126902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126872    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.126902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.126898    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert\") pod \"184f81ce-57ce-447f-bbe0-b72179af0e46\" (UID: \"184f81ce-57ce-447f-bbe0-b72179af0e46\") "
Sep 10 12:11:22.127268 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.127239    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca" (OuterVolumeSpecName: "service-ca") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:22.127407 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.127241    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config" (OuterVolumeSpecName: "console-config") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:22.127407 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.127320    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:22.129131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.129108    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:22.129215 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.129142    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh" (OuterVolumeSpecName: "kube-api-access-b7jzh") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "kube-api-access-b7jzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:11:22.129215 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.129168    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "184f81ce-57ce-447f-bbe0-b72179af0e46" (UID: "184f81ce-57ce-447f-bbe0-b72179af0e46"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:22.228313 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228283    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.228313 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228310    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/184f81ce-57ce-447f-bbe0-b72179af0e46-console-oauth-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.228493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228320    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7jzh\" (UniqueName: \"kubernetes.io/projected/184f81ce-57ce-447f-bbe0-b72179af0e46-kube-api-access-b7jzh\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.228493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228330    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.228493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228340    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-console-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.228493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.228348    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/184f81ce-57ce-447f-bbe0-b72179af0e46-oauth-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:22.363076 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.363042    2551 generic.go:356] "Generic (PLEG): container finished" podID="184f81ce-57ce-447f-bbe0-b72179af0e46" containerID="d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57" exitCode=0
Sep 10 12:11:22.363259 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.363086    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f667dcc4-2cxtk"
Sep 10 12:11:22.363259 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.363094    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f667dcc4-2cxtk" event={"ID":"184f81ce-57ce-447f-bbe0-b72179af0e46","Type":"ContainerDied","Data":"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"}
Sep 10 12:11:22.363259 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.363132    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f667dcc4-2cxtk" event={"ID":"184f81ce-57ce-447f-bbe0-b72179af0e46","Type":"ContainerDied","Data":"ef2c4aa724f42d18d77b516afacaed18ca03a7784593028255cf148a664abfa9"}
Sep 10 12:11:22.363259 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.363151    2551 scope.go:117] "RemoveContainer" containerID="d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"
Sep 10 12:11:22.373198 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.373163    2551 scope.go:117] "RemoveContainer" containerID="d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"
Sep 10 12:11:22.373537 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:11:22.373510    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57\": container with ID starting with d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57 not found: ID does not exist" containerID="d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"
Sep 10 12:11:22.373626 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.373542    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57"} err="failed to get container status \"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57\": rpc error: code = NotFound desc = could not find container \"d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57\": container with ID starting with d182c5343411a7fc6838fb3329a179d97a186426b206d771969b3b4715870f57 not found: ID does not exist"
Sep 10 12:11:22.385220 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.385193    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:11:22.391149 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.391125    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f667dcc4-2cxtk"]
Sep 10 12:11:22.722645 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:22.722553    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="184f81ce-57ce-447f-bbe0-b72179af0e46" path="/var/lib/kubelet/pods/184f81ce-57ce-447f-bbe0-b72179af0e46/volumes"
Sep 10 12:11:30.592971 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.592931    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerName="registry" containerID="cri-o://dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d" gracePeriod=30
Sep 10 12:11:30.861436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.861362    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:11:30.907877 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.907849    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.907913    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.907937    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.907957    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.907973    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.908001    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kndhr\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908276 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.908031    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908276 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.908074    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets\") pod \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\" (UID: \"ab51ca46-1fb7-44f1-8b8e-1bab82772093\") "
Sep 10 12:11:30.908467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.908430    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:30.908872 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.908842    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:30.910771 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.910729    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:30.910891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.910802    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:11:30.910891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.910838    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:30.911161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.911139    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:11:30.911487 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.911467    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr" (OuterVolumeSpecName: "kube-api-access-kndhr") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "kube-api-access-kndhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:11:30.916121 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:30.916097    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ab51ca46-1fb7-44f1-8b8e-1bab82772093" (UID: "ab51ca46-1fb7-44f1-8b8e-1bab82772093"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:11:31.009177 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009151    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009177 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009173    2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-registry-certificates\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009184    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab51ca46-1fb7-44f1-8b8e-1bab82772093-trusted-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009193    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kndhr\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-kube-api-access-kndhr\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009201    2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-image-registry-private-configuration\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009210    2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab51ca46-1fb7-44f1-8b8e-1bab82772093-installation-pull-secrets\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009218    2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab51ca46-1fb7-44f1-8b8e-1bab82772093-ca-trust-extracted\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.009309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.009227    2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab51ca46-1fb7-44f1-8b8e-1bab82772093-bound-sa-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:31.403442 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.403405    2551 generic.go:356] "Generic (PLEG): container finished" podID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerID="dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d" exitCode=0
Sep 10 12:11:31.403623 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.403476    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk"
Sep 10 12:11:31.403623 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.403487    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" event={"ID":"ab51ca46-1fb7-44f1-8b8e-1bab82772093","Type":"ContainerDied","Data":"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"}
Sep 10 12:11:31.403623 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.403559    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679f9b85f8-v64kk" event={"ID":"ab51ca46-1fb7-44f1-8b8e-1bab82772093","Type":"ContainerDied","Data":"b6b050ac02e83081c69d404c3538de385bd5ea1f73b24e6b0ed7702eb5eda403"}
Sep 10 12:11:31.403623 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.403579    2551 scope.go:117] "RemoveContainer" containerID="dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"
Sep 10 12:11:31.413617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.413598    2551 scope.go:117] "RemoveContainer" containerID="dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"
Sep 10 12:11:31.413884 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:11:31.413863    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d\": container with ID starting with dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d not found: ID does not exist" containerID="dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"
Sep 10 12:11:31.413947 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.413896    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d"} err="failed to get container status \"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d\": rpc error: code = NotFound desc = could not find container \"dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d\": container with ID starting with dc78cbcbdca9c298b8952a8f1b7c74e63778a6f1a29bb2083d89084ec1f65e5d not found: ID does not exist"
Sep 10 12:11:31.424534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.424513    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:11:31.427961 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:31.427939    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-679f9b85f8-v64kk"]
Sep 10 12:11:32.721839 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:32.721815    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" path="/var/lib/kubelet/pods/ab51ca46-1fb7-44f1-8b8e-1bab82772093/volumes"
Sep 10 12:11:34.805545 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:34.805510    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-598cd55b69-gmcwr" podUID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" containerName="console" containerID="cri-o://433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc" gracePeriod=15
Sep 10 12:11:35.093077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.093051    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:35.249099 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249072    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249115    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249179    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249212    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249435 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249411    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249457    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkx2v\" (UniqueName: \"kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249486    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert\") pod \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\" (UID: \"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426\") "
Sep 10 12:11:35.249607 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249517    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:35.249607 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249573    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config" (OuterVolumeSpecName: "console-config") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:35.249821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249794    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:35.249821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249804    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:11:35.249950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249821    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.249950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.249839    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.251431 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.251406    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:35.251530 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.251449    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v" (OuterVolumeSpecName: "kube-api-access-gkx2v") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "kube-api-access-gkx2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:11:35.251530 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.251464    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" (UID: "fb5bb786-eac8-4e21-b9dd-bfd1a21f0426"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:11:35.350991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.350941    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-oauth-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.350991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.350960    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-trusted-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.350991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.350970    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkx2v\" (UniqueName: \"kubernetes.io/projected/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-kube-api-access-gkx2v\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.350991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.350980    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-oauth-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.350991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.350988    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426-console-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:11:35.421997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.421971    2551 generic.go:356] "Generic (PLEG): container finished" podID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" containerID="433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc" exitCode=0
Sep 10 12:11:35.422090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.422025    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598cd55b69-gmcwr"
Sep 10 12:11:35.422090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.422058    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598cd55b69-gmcwr" event={"ID":"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426","Type":"ContainerDied","Data":"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"}
Sep 10 12:11:35.422190 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.422106    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598cd55b69-gmcwr" event={"ID":"fb5bb786-eac8-4e21-b9dd-bfd1a21f0426","Type":"ContainerDied","Data":"70d367f98c690e07b2215014bdcf97ca4a50107249802d5aa8fe3284004d6316"}
Sep 10 12:11:35.422190 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.422125    2551 scope.go:117] "RemoveContainer" containerID="433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"
Sep 10 12:11:35.430548 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.430532    2551 scope.go:117] "RemoveContainer" containerID="433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"
Sep 10 12:11:35.430785 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:11:35.430768    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc\": container with ID starting with 433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc not found: ID does not exist" containerID="433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"
Sep 10 12:11:35.430836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.430793    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc"} err="failed to get container status \"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc\": rpc error: code = NotFound desc = could not find container \"433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc\": container with ID starting with 433d1b7cb6922e558d88aa829d05c20922efff676846fdb257e7c8c9e96550bc not found: ID does not exist"
Sep 10 12:11:35.443045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.443024    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:35.446619 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:35.446600    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-598cd55b69-gmcwr"]
Sep 10 12:11:36.428662 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:36.428638    2551 generic.go:356] "Generic (PLEG): container finished" podID="90fb8de4-a0c3-4250-bca8-44c17293d423" containerID="454b8c8f2df16c492d392f7e54a48515121aad0410ed6d12e7071dec7177edd0" exitCode=0
Sep 10 12:11:36.429033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:36.428709    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl" event={"ID":"90fb8de4-a0c3-4250-bca8-44c17293d423","Type":"ContainerDied","Data":"454b8c8f2df16c492d392f7e54a48515121aad0410ed6d12e7071dec7177edd0"}
Sep 10 12:11:36.429033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:36.428958    2551 scope.go:117] "RemoveContainer" containerID="454b8c8f2df16c492d392f7e54a48515121aad0410ed6d12e7071dec7177edd0"
Sep 10 12:11:36.722002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:36.721932    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" path="/var/lib/kubelet/pods/fb5bb786-eac8-4e21-b9dd-bfd1a21f0426/volumes"
Sep 10 12:11:37.434337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:37.434305    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-jxnnl" event={"ID":"90fb8de4-a0c3-4250-bca8-44c17293d423","Type":"ContainerStarted","Data":"79de5505db08ea1a6aef231cf86a3d8cd4ea924503d19f76d6de5e4219575503"}
Sep 10 12:11:40.447597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:40.447567    2551 generic.go:356] "Generic (PLEG): container finished" podID="40af8ac6-a203-4ac5-8e54-64baac884dda" containerID="5c9cc418f131e37bad99a5a8e2457c79219cbaf593f56fa4637ee69ad8d954ea" exitCode=0
Sep 10 12:11:40.447925 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:40.447629    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq" event={"ID":"40af8ac6-a203-4ac5-8e54-64baac884dda","Type":"ContainerDied","Data":"5c9cc418f131e37bad99a5a8e2457c79219cbaf593f56fa4637ee69ad8d954ea"}
Sep 10 12:11:40.447968 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:40.447938    2551 scope.go:117] "RemoveContainer" containerID="5c9cc418f131e37bad99a5a8e2457c79219cbaf593f56fa4637ee69ad8d954ea"
Sep 10 12:11:41.453511 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:41.453475    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-tgczq" event={"ID":"40af8ac6-a203-4ac5-8e54-64baac884dda","Type":"ContainerStarted","Data":"c2c423196d1f5c0f7a5db5a77d220d43b5f90367f4d077d3a06148d48f155640"}
Sep 10 12:11:46.475395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:46.475352    2551 generic.go:356] "Generic (PLEG): container finished" podID="886da443-9220-47dc-9222-7b1aca3a6e78" containerID="ae8a41942ab90adc32e7100ea1f5295cc50771328e2eb821ec8adf563caabf81" exitCode=0
Sep 10 12:11:46.475751 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:46.475408    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv" event={"ID":"886da443-9220-47dc-9222-7b1aca3a6e78","Type":"ContainerDied","Data":"ae8a41942ab90adc32e7100ea1f5295cc50771328e2eb821ec8adf563caabf81"}
Sep 10 12:11:46.475751 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:46.475691    2551 scope.go:117] "RemoveContainer" containerID="ae8a41942ab90adc32e7100ea1f5295cc50771328e2eb821ec8adf563caabf81"
Sep 10 12:11:47.482320 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:11:47.482289    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zx2lv" event={"ID":"886da443-9220-47dc-9222-7b1aca3a6e78","Type":"ContainerStarted","Data":"364c02d97160007ccddc7eacad593159566b6b835c797922c56b8bde9c495689"}
Sep 10 12:12:37.138240 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:12:37.138188    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:13:02.159035 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.158968    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-765f5b8846-wfpnc" podUID="37dfeea0-670a-493f-b04f-1c5cca100296" containerName="console" containerID="cri-o://b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561" gracePeriod=15
Sep 10 12:13:02.398881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.398858    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:13:02.425000 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.424933    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425000 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.424975    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425035    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4qqs\" (UniqueName: \"kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425062    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425080    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425097    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425176 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425136    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert\") pod \"37dfeea0-670a-493f-b04f-1c5cca100296\" (UID: \"37dfeea0-670a-493f-b04f-1c5cca100296\") "
Sep 10 12:13:02.425675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425639    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca" (OuterVolumeSpecName: "service-ca") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:02.425675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425650    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config" (OuterVolumeSpecName: "console-config") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:02.425852 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425770    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:02.425911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.425853    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:02.427609 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.427556    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:13:02.427773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.427750    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:13:02.427773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.427762    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs" (OuterVolumeSpecName: "kube-api-access-l4qqs") pod "37dfeea0-670a-493f-b04f-1c5cca100296" (UID: "37dfeea0-670a-493f-b04f-1c5cca100296"). InnerVolumeSpecName "kube-api-access-l4qqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:13:02.525931 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525910    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4qqs\" (UniqueName: \"kubernetes.io/projected/37dfeea0-670a-493f-b04f-1c5cca100296-kube-api-access-l4qqs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.525931 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525929    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-oauth-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.526048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525938    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-console-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.526048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525946    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-oauth-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.526048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525956    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37dfeea0-670a-493f-b04f-1c5cca100296-console-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.526048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525965    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-trusted-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.526048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.525973    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37dfeea0-670a-493f-b04f-1c5cca100296-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:13:02.773141 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.773114    2551 generic.go:356] "Generic (PLEG): container finished" podID="37dfeea0-670a-493f-b04f-1c5cca100296" containerID="b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561" exitCode=0
Sep 10 12:13:02.773239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.773159    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765f5b8846-wfpnc"
Sep 10 12:13:02.773239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.773198    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765f5b8846-wfpnc" event={"ID":"37dfeea0-670a-493f-b04f-1c5cca100296","Type":"ContainerDied","Data":"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"}
Sep 10 12:13:02.773346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.773241    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765f5b8846-wfpnc" event={"ID":"37dfeea0-670a-493f-b04f-1c5cca100296","Type":"ContainerDied","Data":"525a8243c873a404aed17dfe8ff353bc0c0782b9e4dbf72a58ad4dcacba0aca9"}
Sep 10 12:13:02.773346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.773263    2551 scope.go:117] "RemoveContainer" containerID="b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"
Sep 10 12:13:02.781045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.781029    2551 scope.go:117] "RemoveContainer" containerID="b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"
Sep 10 12:13:02.781303 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:13:02.781285    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561\": container with ID starting with b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561 not found: ID does not exist" containerID="b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"
Sep 10 12:13:02.781346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.781310    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561"} err="failed to get container status \"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561\": rpc error: code = NotFound desc = could not find container \"b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561\": container with ID starting with b8aab1e1dfcc54aae3fbe02b11c75c6e67652dab52fa1106f981bf9713306561 not found: ID does not exist"
Sep 10 12:13:02.789304 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.789283    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:13:02.792698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:02.792679    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-765f5b8846-wfpnc"]
Sep 10 12:13:04.721671 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:13:04.721637    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37dfeea0-670a-493f-b04f-1c5cca100296" path="/var/lib/kubelet/pods/37dfeea0-670a-493f-b04f-1c5cca100296/volumes"
Sep 10 12:14:32.592689 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:32.592659    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:14:32.594546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:32.594520    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:14:32.597687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:32.597663    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:14:32.599835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:32.599812    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:14:32.605061 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:32.605045    2551 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 12:14:42.546613 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546582    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"]
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546852    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546862    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546873    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37dfeea0-670a-493f-b04f-1c5cca100296" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546878    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="37dfeea0-670a-493f-b04f-1c5cca100296" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546887    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184f81ce-57ce-447f-bbe0-b72179af0e46" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546892    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="184f81ce-57ce-447f-bbe0-b72179af0e46" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546907    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerName="registry"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546912    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerName="registry"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546962    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="184f81ce-57ce-447f-bbe0-b72179af0e46" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546969    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab51ca46-1fb7-44f1-8b8e-1bab82772093" containerName="registry"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546975    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="37dfeea0-670a-493f-b04f-1c5cca100296" containerName="console"
Sep 10 12:14:42.548025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.546980    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5bb786-eac8-4e21-b9dd-bfd1a21f0426" containerName="console"
Sep 10 12:14:42.548813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.548799    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.551367 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.551342    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 12:14:42.551367 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.551351    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 12:14:42.552075 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.552060    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 12:14:42.552761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.552744    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 12:14:42.552849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.552791    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wqs45\""
Sep 10 12:14:42.580690 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.580666    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"]
Sep 10 12:14:42.685204 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.685183    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-manifests\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.685298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.685209    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc6t7\" (UniqueName: \"kubernetes.io/projected/5379cddf-005d-477e-bed8-069b0d5d7e25-kube-api-access-kc6t7\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.685298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.685228    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.685298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.685246    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-tmp\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.685420 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.685315    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786285 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786258    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786303    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-manifests\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786320    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc6t7\" (UniqueName: \"kubernetes.io/projected/5379cddf-005d-477e-bed8-069b0d5d7e25-kube-api-access-kc6t7\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786337    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786432 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786362    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-tmp\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.786739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.786718    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-manifests\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.788540 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.788522    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379cddf-005d-477e-bed8-069b0d5d7e25-tmp\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.788668 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.788650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.788731 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.788714    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5379cddf-005d-477e-bed8-069b0d5d7e25-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.799209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.799155    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc6t7\" (UniqueName: \"kubernetes.io/projected/5379cddf-005d-477e-bed8-069b0d5d7e25-kube-api-access-kc6t7\") pod \"opendatahub-operator-controller-manager-5f75798bc7-fkpq6\" (UID: \"5379cddf-005d-477e-bed8-069b0d5d7e25\") " pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:42.858340 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:42.858318    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:43.135815 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:43.135793    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"]
Sep 10 12:14:43.138136 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:14:43.138102    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5379cddf_005d_477e_bed8_069b0d5d7e25.slice/crio-0370c8cbd728b5389951a23032496940c446a93dff5118250395078306c10e25 WatchSource:0}: Error finding container 0370c8cbd728b5389951a23032496940c446a93dff5118250395078306c10e25: Status 404 returned error can't find the container with id 0370c8cbd728b5389951a23032496940c446a93dff5118250395078306c10e25
Sep 10 12:14:43.140222 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:43.140204    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:14:44.149277 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:44.149235    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6" event={"ID":"5379cddf-005d-477e-bed8-069b0d5d7e25","Type":"ContainerStarted","Data":"0370c8cbd728b5389951a23032496940c446a93dff5118250395078306c10e25"}
Sep 10 12:14:46.161676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:46.161643    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6" event={"ID":"5379cddf-005d-477e-bed8-069b0d5d7e25","Type":"ContainerStarted","Data":"2bdd66332609d655a623789fe097912cbc4dd9b2964375d2f036c0c1994fa844"}
Sep 10 12:14:47.167212 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:47.167185    2551 generic.go:356] "Generic (PLEG): container finished" podID="5379cddf-005d-477e-bed8-069b0d5d7e25" containerID="2bdd66332609d655a623789fe097912cbc4dd9b2964375d2f036c0c1994fa844" exitCode=0
Sep 10 12:14:47.171414 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:47.167268    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6" event={"ID":"5379cddf-005d-477e-bed8-069b0d5d7e25","Type":"ContainerDied","Data":"2bdd66332609d655a623789fe097912cbc4dd9b2964375d2f036c0c1994fa844"}
Sep 10 12:14:48.173856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:48.173824    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6" event={"ID":"5379cddf-005d-477e-bed8-069b0d5d7e25","Type":"ContainerStarted","Data":"6b7b0fa19377ca95e69d5fb9fca372ab5efe5443048f3ffd4e15011fbb1b478d"}
Sep 10 12:14:48.174216 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:48.173909    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:14:48.214553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:48.214499    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6" podStartSLOduration=1.896572528 podStartE2EDuration="6.214485636s" totalImagesPullingTime="3.234437472s" totalInitContainerRuntime="1.083475636s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:14:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:14:48.211832262 +0000 UTC m=+315.992389745" watchObservedRunningTime="2026-09-10 12:14:48.214485636 +0000 UTC m=+315.995043109"
Sep 10 12:14:59.181059 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:14:59.181028    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f75798bc7-fkpq6"
Sep 10 12:15:01.847297 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:01.847256    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"]
Sep 10 12:15:01.855055 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:01.855028    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:01.857725 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:01.857700    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 12:15:01.857725 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:01.857718    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 12:15:01.871657 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:01.871637    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"]
Sep 10 12:15:02.034389 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.034354    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p78c\" (UniqueName: \"kubernetes.io/projected/07a8d33f-cdf4-48ef-9a98-650a2be25e02-kube-api-access-6p78c\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.034493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.034434    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tmp\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.034493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.034468    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.135029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.134971    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p78c\" (UniqueName: \"kubernetes.io/projected/07a8d33f-cdf4-48ef-9a98-650a2be25e02-kube-api-access-6p78c\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.135029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.135005    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tmp\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.135029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.135025    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.135188 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:15:02.135147    2551 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 12:15:02.135228 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:15:02.135205    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs podName:07a8d33f-cdf4-48ef-9a98-650a2be25e02 nodeName:}" failed. No retries permitted until 2026-09-10 12:15:02.63518735 +0000 UTC m=+330.415744801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs") pod "kube-auth-proxy-688cb69f77-9llgl" (UID: "07a8d33f-cdf4-48ef-9a98-650a2be25e02") : secret "kube-auth-proxy-tls" not found
Sep 10 12:15:02.137171 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.137150    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tmp\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.147770 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.147746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p78c\" (UniqueName: \"kubernetes.io/projected/07a8d33f-cdf4-48ef-9a98-650a2be25e02-kube-api-access-6p78c\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.640251 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.640219    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.642509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.642490    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a8d33f-cdf4-48ef-9a98-650a2be25e02-tls-certs\") pod \"kube-auth-proxy-688cb69f77-9llgl\" (UID: \"07a8d33f-cdf4-48ef-9a98-650a2be25e02\") " pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:02.764630 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:02.764603    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"
Sep 10 12:15:03.039233 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.039126    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688cb69f77-9llgl"]
Sep 10 12:15:03.041293 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:15:03.041265    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a8d33f_cdf4_48ef_9a98_650a2be25e02.slice/crio-ec7466e72ca94d73d14a8e002c38f834fa08eefbceb6857b2387e0d9a7db7181 WatchSource:0}: Error finding container ec7466e72ca94d73d14a8e002c38f834fa08eefbceb6857b2387e0d9a7db7181: Status 404 returned error can't find the container with id ec7466e72ca94d73d14a8e002c38f834fa08eefbceb6857b2387e0d9a7db7181
Sep 10 12:15:03.235868 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.235836    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl" event={"ID":"07a8d33f-cdf4-48ef-9a98-650a2be25e02","Type":"ContainerStarted","Data":"ec7466e72ca94d73d14a8e002c38f834fa08eefbceb6857b2387e0d9a7db7181"}
Sep 10 12:15:03.298227 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.298169    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:03.301169 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.301152    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.303601 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.303580    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 12:15:03.303601 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.303590    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 12:15:03.303897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.303880    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 12:15:03.303897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.303891    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jvtqq\""
Sep 10 12:15:03.303994 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.303878    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 12:15:03.313366 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.313343    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:03.447145 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447120    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447248 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447151    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dzpw\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447248 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447177    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447248 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447398 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447267    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447398 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.447398 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.447342    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548639 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548588    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548639 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548632    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548766 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548654    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548904 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548890    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548971 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548942    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.548971 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548960    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dzpw\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.549080 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.548976    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.549278 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.549259    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.550996 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.550969    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.551081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.550974    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.551208 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.551193    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.551244 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.551227    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.557183 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.557156    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.557424 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.557408    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dzpw\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw\") pod \"istiod-openshift-gateway-7b7995d5cd-bj9z5\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.610317 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.610296    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:03.878325 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:03.878287    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:03.880308 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:15:03.880284    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452eec72_2ef0_43c0_b496_e546180ed78e.slice/crio-5485b7cfc449c10056f0f5ba8ab051c57de6d6f7565c85028db08c59f49edb34 WatchSource:0}: Error finding container 5485b7cfc449c10056f0f5ba8ab051c57de6d6f7565c85028db08c59f49edb34: Status 404 returned error can't find the container with id 5485b7cfc449c10056f0f5ba8ab051c57de6d6f7565c85028db08c59f49edb34
Sep 10 12:15:04.243674 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:04.243598    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" event={"ID":"452eec72-2ef0-43c0-b496-e546180ed78e","Type":"ContainerStarted","Data":"5485b7cfc449c10056f0f5ba8ab051c57de6d6f7565c85028db08c59f49edb34"}
Sep 10 12:15:06.258741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:06.258679    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl" event={"ID":"07a8d33f-cdf4-48ef-9a98-650a2be25e02","Type":"ContainerStarted","Data":"d53186a562ac6beb9ffbe359dc27d6467ff8e8d052e1c5bcfcd8c7e7af3b2b38"}
Sep 10 12:15:06.278337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:06.278277    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-688cb69f77-9llgl" podStartSLOduration=2.226401555 podStartE2EDuration="5.278257484s" totalImagesPullingTime="3.051855929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:06.277167103 +0000 UTC m=+334.057724612" watchObservedRunningTime="2026-09-10 12:15:06.278257484 +0000 UTC m=+334.058814956"
Sep 10 12:15:07.742750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:07.742717    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 12:15:07.742976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:07.742783    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 12:15:08.269224 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:08.269188    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" event={"ID":"452eec72-2ef0-43c0-b496-e546180ed78e","Type":"ContainerStarted","Data":"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"}
Sep 10 12:15:08.269426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:08.269340    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:08.270785 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:08.270764    2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bj9z5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 12:15:08.270837 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:08.270814    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 12:15:08.295982 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:08.295940    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" podStartSLOduration=1.435643783 podStartE2EDuration="5.295928405s" totalImagesPullingTime="3.860284622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:08.293465448 +0000 UTC m=+336.074022933" watchObservedRunningTime="2026-09-10 12:15:08.295928405 +0000 UTC m=+336.076485878"
Sep 10 12:15:09.273877 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:09.273853    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:24.544698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.544667    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"]
Sep 10 12:15:24.549181 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.549164    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.558108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.558090    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 12:15:24.560365 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.560350    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gstlq\""
Sep 10 12:15:24.590273 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.590251    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"]
Sep 10 12:15:24.605978 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.605956    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpvmj\" (UniqueName: \"kubernetes.io/projected/28a25d9d-363f-4bb2-9770-aa511b9fca5a-kube-api-access-xpvmj\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.606072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.606009    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28a25d9d-363f-4bb2-9770-aa511b9fca5a-cert\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.706931 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.706908    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28a25d9d-363f-4bb2-9770-aa511b9fca5a-cert\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.707022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.706956    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpvmj\" (UniqueName: \"kubernetes.io/projected/28a25d9d-363f-4bb2-9770-aa511b9fca5a-kube-api-access-xpvmj\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.709277 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.709254    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28a25d9d-363f-4bb2-9770-aa511b9fca5a-cert\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.728311 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.728289    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpvmj\" (UniqueName: \"kubernetes.io/projected/28a25d9d-363f-4bb2-9770-aa511b9fca5a-kube-api-access-xpvmj\") pod \"kserve-controller-manager-d6f4b65b8-djwfr\" (UID: \"28a25d9d-363f-4bb2-9770-aa511b9fca5a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.862915 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.862873    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:24.983462 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.983402    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:24.983640 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:24.983605    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" containerName="discovery" containerID="cri-o://f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e" gracePeriod=30
Sep 10 12:15:25.156998 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.156958    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"]
Sep 10 12:15:25.167296 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:15:25.167267    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a25d9d_363f_4bb2_9770_aa511b9fca5a.slice/crio-fdd8764ed7ece34246a35bffa32b38a0f9e074a5ec172a3f60f6d84f2ba82c38 WatchSource:0}: Error finding container fdd8764ed7ece34246a35bffa32b38a0f9e074a5ec172a3f60f6d84f2ba82c38: Status 404 returned error can't find the container with id fdd8764ed7ece34246a35bffa32b38a0f9e074a5ec172a3f60f6d84f2ba82c38
Sep 10 12:15:25.212097 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.212076    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:25.312969 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.312943    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.312978    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dzpw\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313038    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313064    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313078    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313095    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313109    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap\") pod \"452eec72-2ef0-43c0-b496-e546180ed78e\" (UID: \"452eec72-2ef0-43c0-b496-e546180ed78e\") "
Sep 10 12:15:25.313650 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.313624    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:15:25.315364 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315343    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:15:25.315471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315334    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts" (OuterVolumeSpecName: "cacerts") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:15:25.315527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315475    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs" (OuterVolumeSpecName: "local-certs") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:15:25.315527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315494    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw" (OuterVolumeSpecName: "kube-api-access-7dzpw") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "kube-api-access-7dzpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:15:25.315598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315571    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:15:25.315598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.315573    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token" (OuterVolumeSpecName: "istio-token") pod "452eec72-2ef0-43c0-b496-e546180ed78e" (UID: "452eec72-2ef0-43c0-b496-e546180ed78e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:15:25.348967 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.348940    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr" event={"ID":"28a25d9d-363f-4bb2-9770-aa511b9fca5a","Type":"ContainerStarted","Data":"fdd8764ed7ece34246a35bffa32b38a0f9e074a5ec172a3f60f6d84f2ba82c38"}
Sep 10 12:15:25.350129 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.350110    2551 generic.go:356] "Generic (PLEG): container finished" podID="452eec72-2ef0-43c0-b496-e546180ed78e" containerID="f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e" exitCode=0
Sep 10 12:15:25.350201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.350148    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"
Sep 10 12:15:25.350201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.350157    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" event={"ID":"452eec72-2ef0-43c0-b496-e546180ed78e","Type":"ContainerDied","Data":"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"}
Sep 10 12:15:25.350201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.350187    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5" event={"ID":"452eec72-2ef0-43c0-b496-e546180ed78e","Type":"ContainerDied","Data":"5485b7cfc449c10056f0f5ba8ab051c57de6d6f7565c85028db08c59f49edb34"}
Sep 10 12:15:25.350301 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.350207    2551 scope.go:117] "RemoveContainer" containerID="f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"
Sep 10 12:15:25.358629 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.358611    2551 scope.go:117] "RemoveContainer" containerID="f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"
Sep 10 12:15:25.358931 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:15:25.358912    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e\": container with ID starting with f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e not found: ID does not exist" containerID="f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"
Sep 10 12:15:25.358997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.358936    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e"} err="failed to get container status \"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e\": rpc error: code = NotFound desc = could not find container \"f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e\": container with ID starting with f0c7365d4169fc64a167e93c6513d18e961d7b415870e0124b0fcbd6a8de341e not found: ID does not exist"
Sep 10 12:15:25.403236 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.403190    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:25.410412 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.410389    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bj9z5"]
Sep 10 12:15:25.413987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.413962    2551 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-kubeconfig\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.413987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.413984    2551 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-cacerts\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.414108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.413996    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.414108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.414007    2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-ca-configmap\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.414108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.414020    2551 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/452eec72-2ef0-43c0-b496-e546180ed78e-local-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.414108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.414033    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dzpw\" (UniqueName: \"kubernetes.io/projected/452eec72-2ef0-43c0-b496-e546180ed78e-kube-api-access-7dzpw\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:25.414108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:25.414044    2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/452eec72-2ef0-43c0-b496-e546180ed78e-istio-csr-dns-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:15:26.723391 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:26.723348    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" path="/var/lib/kubelet/pods/452eec72-2ef0-43c0-b496-e546180ed78e/volumes"
Sep 10 12:15:27.362388 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:27.362334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr" event={"ID":"28a25d9d-363f-4bb2-9770-aa511b9fca5a","Type":"ContainerStarted","Data":"d50d00367de4f8f48000e23086d22184d11fbff0c9c28433fe750738fadc689b"}
Sep 10 12:15:27.362527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:27.362430    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:15:27.402545 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:27.402502    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr" podStartSLOduration=1.346002607 podStartE2EDuration="3.402487871s" totalImagesPullingTime="2.056485264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:27.390078034 +0000 UTC m=+355.170635507" watchObservedRunningTime="2026-09-10 12:15:27.402487871 +0000 UTC m=+355.183045347"
Sep 10 12:15:46.858985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.858947    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86845cf594-bghc4"]
Sep 10 12:15:46.859560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.859406    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" containerName="discovery"
Sep 10 12:15:46.859560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.859425    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" containerName="discovery"
Sep 10 12:15:46.859560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.859513    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="452eec72-2ef0-43c0-b496-e546180ed78e" containerName="discovery"
Sep 10 12:15:46.867977 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.867954    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.880000 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.879982    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 12:15:46.881147 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.881130    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 12:15:46.881202 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.881162    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 12:15:46.881293 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.881270    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4h9f5\""
Sep 10 12:15:46.881493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.881478    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 12:15:46.886675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.886655    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86845cf594-bghc4"]
Sep 10 12:15:46.890024 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.890001    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 12:15:46.893825 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.893795    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 12:15:46.973691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973667    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973696    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-trusted-ca-bundle\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973711    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rhx\" (UniqueName: \"kubernetes.io/projected/b0197622-b527-43c2-a846-7c03c54fa12e-kube-api-access-b9rhx\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973732    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-oauth-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973941 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-service-ca\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973941 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-console-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:46.973941 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:46.973880    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-oauth-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074653 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074620    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-oauth-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074653 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074656    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-service-ca\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074803 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074674    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-console-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074862 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074817    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-oauth-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074926 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074910    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074950    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-trusted-ca-bundle\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.074981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.074973    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rhx\" (UniqueName: \"kubernetes.io/projected/b0197622-b527-43c2-a846-7c03c54fa12e-kube-api-access-b9rhx\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.075420 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.075398    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-console-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.075505 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.075420    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-oauth-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.075563 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.075532    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-service-ca\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.075719 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.075701    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0197622-b527-43c2-a846-7c03c54fa12e-trusted-ca-bundle\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.077276 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.077255    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-oauth-config\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.077349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.077309    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0197622-b527-43c2-a846-7c03c54fa12e-console-serving-cert\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.100517 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.100493    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rhx\" (UniqueName: \"kubernetes.io/projected/b0197622-b527-43c2-a846-7c03c54fa12e-kube-api-access-b9rhx\") pod \"console-86845cf594-bghc4\" (UID: \"b0197622-b527-43c2-a846-7c03c54fa12e\") " pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.176275 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.176217    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:47.444701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:47.444548    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86845cf594-bghc4"]
Sep 10 12:15:47.447295 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:15:47.447268    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0197622_b527_43c2_a846_7c03c54fa12e.slice/crio-c6bd0322ca5a89d5db13a7d01c7615104f2f48f86824e4b50809326faf922709 WatchSource:0}: Error finding container c6bd0322ca5a89d5db13a7d01c7615104f2f48f86824e4b50809326faf922709: Status 404 returned error can't find the container with id c6bd0322ca5a89d5db13a7d01c7615104f2f48f86824e4b50809326faf922709
Sep 10 12:15:48.455046 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:48.455011    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86845cf594-bghc4" event={"ID":"b0197622-b527-43c2-a846-7c03c54fa12e","Type":"ContainerStarted","Data":"667404afd29a2b5e9e7c293b0a03104e625798c951b68667a9de0727a681e467"}
Sep 10 12:15:48.455422 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:48.455071    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86845cf594-bghc4" event={"ID":"b0197622-b527-43c2-a846-7c03c54fa12e","Type":"ContainerStarted","Data":"c6bd0322ca5a89d5db13a7d01c7615104f2f48f86824e4b50809326faf922709"}
Sep 10 12:15:48.475849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:48.475802    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86845cf594-bghc4" podStartSLOduration=2.475790264 podStartE2EDuration="2.475790264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:48.474833175 +0000 UTC m=+376.255390671" watchObservedRunningTime="2026-09-10 12:15:48.475790264 +0000 UTC m=+376.256347736"
Sep 10 12:15:57.176718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:57.176686    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:57.177189 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:57.176736    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:57.181023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:57.181004    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:57.495611 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:57.495557    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86845cf594-bghc4"
Sep 10 12:15:58.371366 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:15:58.371339    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-djwfr"
Sep 10 12:16:56.930753 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.930681    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fwfsv"]
Sep 10 12:16:56.933955 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.933933    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:16:56.937241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.937225    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 12:16:56.937491 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.937474    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 12:16:56.937579 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.937503    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-r5mc7\""
Sep 10 12:16:56.958254 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.958231    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fwfsv"]
Sep 10 12:16:56.981884 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:56.981858    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzjt6\" (UniqueName: \"kubernetes.io/projected/47f123ed-b819-45f6-aab7-0b3e019e4ae1-kube-api-access-vzjt6\") pod \"authorino-operator-76b478bd78-fwfsv\" (UID: \"47f123ed-b819-45f6-aab7-0b3e019e4ae1\") " pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:16:57.082662 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:57.082639    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzjt6\" (UniqueName: \"kubernetes.io/projected/47f123ed-b819-45f6-aab7-0b3e019e4ae1-kube-api-access-vzjt6\") pod \"authorino-operator-76b478bd78-fwfsv\" (UID: \"47f123ed-b819-45f6-aab7-0b3e019e4ae1\") " pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:16:57.096131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:57.096107    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzjt6\" (UniqueName: \"kubernetes.io/projected/47f123ed-b819-45f6-aab7-0b3e019e4ae1-kube-api-access-vzjt6\") pod \"authorino-operator-76b478bd78-fwfsv\" (UID: \"47f123ed-b819-45f6-aab7-0b3e019e4ae1\") " pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:16:57.243608 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:57.243547    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:16:57.515842 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:57.515772    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fwfsv"]
Sep 10 12:16:57.519514 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:16:57.519484    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f123ed_b819_45f6_aab7_0b3e019e4ae1.slice/crio-32282b30dcd4a13bc5bdc91343c5f604c342361e2861877cf2846adaa0eb75d9 WatchSource:0}: Error finding container 32282b30dcd4a13bc5bdc91343c5f604c342361e2861877cf2846adaa0eb75d9: Status 404 returned error can't find the container with id 32282b30dcd4a13bc5bdc91343c5f604c342361e2861877cf2846adaa0eb75d9
Sep 10 12:16:57.755239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:16:57.755213    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv" event={"ID":"47f123ed-b819-45f6-aab7-0b3e019e4ae1","Type":"ContainerStarted","Data":"32282b30dcd4a13bc5bdc91343c5f604c342361e2861877cf2846adaa0eb75d9"}
Sep 10 12:17:00.769035 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:00.768966    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv" event={"ID":"47f123ed-b819-45f6-aab7-0b3e019e4ae1","Type":"ContainerStarted","Data":"789967dbde4855aa399ae4d3dff2aaf1669b7d00b306467bad44f99a3c66b011"}
Sep 10 12:17:00.769417 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:00.769109    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:17:00.786713 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:00.786667    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv" podStartSLOduration=1.8560536010000002 podStartE2EDuration="4.786649928s" totalImagesPullingTime="2.930596327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:00.785937098 +0000 UTC m=+448.566494570" watchObservedRunningTime="2026-09-10 12:17:00.786649928 +0000 UTC m=+448.567207401"
Sep 10 12:17:01.253624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.253594    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"]
Sep 10 12:17:01.257022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.257006    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.259748 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.259731    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zjzsx\""
Sep 10 12:17:01.271464 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.271442    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"]
Sep 10 12:17:01.313400 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.313370    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.313487 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.313420    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65ldt\" (UniqueName: \"kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.414117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.414090    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.414207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.414131    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65ldt\" (UniqueName: \"kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.414481 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.414463    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.427239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.427220    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65ldt\" (UniqueName: \"kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.566611 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.566552    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:01.847485 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:01.847463    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"]
Sep 10 12:17:01.849545 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:01.849518    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb961b4e1_a554_4278_9c9f_66bac7f39925.slice/crio-746af603a5cca6c8d9c3cee6e08d67c331f4704e810e058bba4afd0bb2a27fab WatchSource:0}: Error finding container 746af603a5cca6c8d9c3cee6e08d67c331f4704e810e058bba4afd0bb2a27fab: Status 404 returned error can't find the container with id 746af603a5cca6c8d9c3cee6e08d67c331f4704e810e058bba4afd0bb2a27fab
Sep 10 12:17:02.783894 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:02.783862    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr" event={"ID":"b961b4e1-a554-4278-9c9f-66bac7f39925","Type":"ContainerStarted","Data":"746af603a5cca6c8d9c3cee6e08d67c331f4704e810e058bba4afd0bb2a27fab"}
Sep 10 12:17:03.492964 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.492928    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"]
Sep 10 12:17:03.496422 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.496396    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:03.499424 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.499400    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 12:17:03.499578 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.499553    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bwpwv\""
Sep 10 12:17:03.509838 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.509815    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"]
Sep 10 12:17:03.531127 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.531089    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb2xd\" (UniqueName: \"kubernetes.io/projected/40b253a3-dc01-459b-b4b4-9653b799491b-kube-api-access-hb2xd\") pod \"dns-operator-controller-manager-7b84cd7c9c-cnm8k\" (UID: \"40b253a3-dc01-459b-b4b4-9653b799491b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:03.631772 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.631740    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb2xd\" (UniqueName: \"kubernetes.io/projected/40b253a3-dc01-459b-b4b4-9653b799491b-kube-api-access-hb2xd\") pod \"dns-operator-controller-manager-7b84cd7c9c-cnm8k\" (UID: \"40b253a3-dc01-459b-b4b4-9653b799491b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:03.642944 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.642924    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb2xd\" (UniqueName: \"kubernetes.io/projected/40b253a3-dc01-459b-b4b4-9653b799491b-kube-api-access-hb2xd\") pod \"dns-operator-controller-manager-7b84cd7c9c-cnm8k\" (UID: \"40b253a3-dc01-459b-b4b4-9653b799491b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:03.808822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:03.808722    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:04.092793 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:04.092764    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"]
Sep 10 12:17:04.094468 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:04.094434    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b253a3_dc01_459b_b4b4_9653b799491b.slice/crio-e1fa8b3d632dfdab5261a356c823e2512cccc26aeed1f2258de51e81e9b2c458 WatchSource:0}: Error finding container e1fa8b3d632dfdab5261a356c823e2512cccc26aeed1f2258de51e81e9b2c458: Status 404 returned error can't find the container with id e1fa8b3d632dfdab5261a356c823e2512cccc26aeed1f2258de51e81e9b2c458
Sep 10 12:17:04.803705 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:04.803669    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k" event={"ID":"40b253a3-dc01-459b-b4b4-9653b799491b","Type":"ContainerStarted","Data":"e1fa8b3d632dfdab5261a356c823e2512cccc26aeed1f2258de51e81e9b2c458"}
Sep 10 12:17:05.496659 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.496631    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"]
Sep 10 12:17:05.499915 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.499898    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:05.502403 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.502368    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mlrsm\""
Sep 10 12:17:05.511487 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.511468    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"]
Sep 10 12:17:05.548325 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.548299    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjhft\" (UniqueName: \"kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft\") pod \"limitador-operator-controller-manager-74495c69f8-7xqqz\" (UID: \"a80b16e8-65bf-4027-b195-3007e5221b21\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:05.649586 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.649533    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjhft\" (UniqueName: \"kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft\") pod \"limitador-operator-controller-manager-74495c69f8-7xqqz\" (UID: \"a80b16e8-65bf-4027-b195-3007e5221b21\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:05.659887 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.659862    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjhft\" (UniqueName: \"kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft\") pod \"limitador-operator-controller-manager-74495c69f8-7xqqz\" (UID: \"a80b16e8-65bf-4027-b195-3007e5221b21\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:05.808702 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.808676    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:05.813168 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.813139    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr" event={"ID":"b961b4e1-a554-4278-9c9f-66bac7f39925","Type":"ContainerStarted","Data":"70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325"}
Sep 10 12:17:05.813261 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.813245    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:05.854056 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:05.853998    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr" podStartSLOduration=1.340665856 podStartE2EDuration="4.85397912s" totalImagesPullingTime="3.513313264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:05.850806468 +0000 UTC m=+453.631363960" watchObservedRunningTime="2026-09-10 12:17:05.85397912 +0000 UTC m=+453.634536593"
Sep 10 12:17:06.081162 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:06.081136    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"]
Sep 10 12:17:06.083995 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:06.083962    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80b16e8_65bf_4027_b195_3007e5221b21.slice/crio-a745c3636e16e70c2f20476b070b8a5c2d55300025657d857ed5810f1a58567c WatchSource:0}: Error finding container a745c3636e16e70c2f20476b070b8a5c2d55300025657d857ed5810f1a58567c: Status 404 returned error can't find the container with id a745c3636e16e70c2f20476b070b8a5c2d55300025657d857ed5810f1a58567c
Sep 10 12:17:06.818842 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:06.818811    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz" event={"ID":"a80b16e8-65bf-4027-b195-3007e5221b21","Type":"ContainerStarted","Data":"a745c3636e16e70c2f20476b070b8a5c2d55300025657d857ed5810f1a58567c"}
Sep 10 12:17:09.834235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:09.834198    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz" event={"ID":"a80b16e8-65bf-4027-b195-3007e5221b21","Type":"ContainerStarted","Data":"4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97"}
Sep 10 12:17:09.834603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:09.834263    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:09.854614 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:09.854564    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz" podStartSLOduration=1.6462479490000002 podStartE2EDuration="4.854547284s" totalImagesPullingTime="3.208299335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:09.851769241 +0000 UTC m=+457.632326742" watchObservedRunningTime="2026-09-10 12:17:09.854547284 +0000 UTC m=+457.635104755"
Sep 10 12:17:11.779360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:11.779327    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-fwfsv"
Sep 10 12:17:15.862433 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:15.862401    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k" event={"ID":"40b253a3-dc01-459b-b4b4-9653b799491b","Type":"ContainerStarted","Data":"2c67397d921f5bbc329af186d6ebc3d766f23f89942b736d75aa16c520ce1571"}
Sep 10 12:17:15.862775 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:15.862476    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:15.903384 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:15.903328    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k" podStartSLOduration=1.335845275 podStartE2EDuration="12.903311013s" totalImagesPullingTime="11.567465738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:15.900176559 +0000 UTC m=+463.680734031" watchObservedRunningTime="2026-09-10 12:17:15.903311013 +0000 UTC m=+463.683868488"
Sep 10 12:17:16.821624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:16.821599    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.332604 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.332574    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"]
Sep 10 12:17:19.332977 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.332756    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" containerName="manager" containerID="cri-o://70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325" gracePeriod=2
Sep 10 12:17:19.351746 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.351720    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"]
Sep 10 12:17:19.369467 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.369424    2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b961b4e1-a554-4278-9c9f-66bac7f39925\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\": pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.379013 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.378986    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:19.379405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.379367    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" containerName="manager"
Sep 10 12:17:19.379475 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.379409    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" containerName="manager"
Sep 10 12:17:19.379560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.379546    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" containerName="manager"
Sep 10 12:17:19.382500 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.382481    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"]
Sep 10 12:17:19.382633 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.382611    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.382696 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.382659    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" containerName="manager" containerID="cri-o://4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97" gracePeriod=2
Sep 10 12:17:19.384317 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.384296    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.385328 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.385291    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.406351 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.406326    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:19.407770 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.407750    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"]
Sep 10 12:17:19.421729 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.421704    2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a80b16e8-65bf-4027-b195-3007e5221b21\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-7xqqz\": pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.423853 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.423835    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"]
Sep 10 12:17:19.424141 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.424129    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" containerName="manager"
Sep 10 12:17:19.424200 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.424141    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" containerName="manager"
Sep 10 12:17:19.424250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.424206    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" containerName="manager"
Sep 10 12:17:19.426971 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.426956    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:19.453914 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.453895    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"]
Sep 10 12:17:19.454263 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.454240    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.454402 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.454282    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pld5b\" (UniqueName: \"kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.465519 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.465493    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.475283 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.475256    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.555421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.554953    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6xwb\" (UniqueName: \"kubernetes.io/projected/773b95ef-0eef-411d-88e9-7688a1f71055-kube-api-access-d6xwb\") pod \"limitador-operator-controller-manager-74495c69f8-jwxxq\" (UID: \"773b95ef-0eef-411d-88e9-7688a1f71055\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:19.555421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.555035    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.555421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.555069    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pld5b\" (UniqueName: \"kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.555421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.555362    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.568989 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.567192    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pld5b\" (UniqueName: \"kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hs5cp\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.615293 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.615269    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.622199 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.622173    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.624715 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.624686    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.631696 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.631680    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.634135 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.634114    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.636580 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.636560    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.656113 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.656094    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6xwb\" (UniqueName: \"kubernetes.io/projected/773b95ef-0eef-411d-88e9-7688a1f71055-kube-api-access-d6xwb\") pod \"limitador-operator-controller-manager-74495c69f8-jwxxq\" (UID: \"773b95ef-0eef-411d-88e9-7688a1f71055\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:19.679593 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.679573    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6xwb\" (UniqueName: \"kubernetes.io/projected/773b95ef-0eef-411d-88e9-7688a1f71055-kube-api-access-d6xwb\") pod \"limitador-operator-controller-manager-74495c69f8-jwxxq\" (UID: \"773b95ef-0eef-411d-88e9-7688a1f71055\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:19.753116 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.753094    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:19.756759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.756742    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjhft\" (UniqueName: \"kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft\") pod \"a80b16e8-65bf-4027-b195-3007e5221b21\" (UID: \"a80b16e8-65bf-4027-b195-3007e5221b21\") "
Sep 10 12:17:19.756849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.756808    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume\") pod \"b961b4e1-a554-4278-9c9f-66bac7f39925\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") "
Sep 10 12:17:19.756849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.756847    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65ldt\" (UniqueName: \"kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt\") pod \"b961b4e1-a554-4278-9c9f-66bac7f39925\" (UID: \"b961b4e1-a554-4278-9c9f-66bac7f39925\") "
Sep 10 12:17:19.757280 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.757257    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b961b4e1-a554-4278-9c9f-66bac7f39925" (UID: "b961b4e1-a554-4278-9c9f-66bac7f39925"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:19.758800 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.758780    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:19.758897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.758779    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft" (OuterVolumeSpecName: "kube-api-access-bjhft") pod "a80b16e8-65bf-4027-b195-3007e5221b21" (UID: "a80b16e8-65bf-4027-b195-3007e5221b21"). InnerVolumeSpecName "kube-api-access-bjhft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:19.758975 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.758818    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt" (OuterVolumeSpecName: "kube-api-access-65ldt") pod "b961b4e1-a554-4278-9c9f-66bac7f39925" (UID: "b961b4e1-a554-4278-9c9f-66bac7f39925"). InnerVolumeSpecName "kube-api-access-65ldt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:19.857966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.857914    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65ldt\" (UniqueName: \"kubernetes.io/projected/b961b4e1-a554-4278-9c9f-66bac7f39925-kube-api-access-65ldt\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:19.857966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.857938    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjhft\" (UniqueName: \"kubernetes.io/projected/a80b16e8-65bf-4027-b195-3007e5221b21-kube-api-access-bjhft\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:19.857966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.857947    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b961b4e1-a554-4278-9c9f-66bac7f39925-extensions-socket-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:19.881250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.881227    2551 generic.go:356] "Generic (PLEG): container finished" podID="a80b16e8-65bf-4027-b195-3007e5221b21" containerID="4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97" exitCode=0
Sep 10 12:17:19.881346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.881269    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.881346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.881317    2551 scope.go:117] "RemoveContainer" containerID="4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97"
Sep 10 12:17:19.882789 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.882761    2551 generic.go:356] "Generic (PLEG): container finished" podID="b961b4e1-a554-4278-9c9f-66bac7f39925" containerID="70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325" exitCode=0
Sep 10 12:17:19.882877 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.882809    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.884279 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.884250    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.886228 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.886208    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.888940 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.888917    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.889581 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.889558    2551 scope.go:117] "RemoveContainer" containerID="4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97"
Sep 10 12:17:19.889817 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.889797    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97\": container with ID starting with 4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97 not found: ID does not exist" containerID="4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97"
Sep 10 12:17:19.889885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.889828    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97"} err="failed to get container status \"4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97\": rpc error: code = NotFound desc = could not find container \"4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97\": container with ID starting with 4f6b22cdf2d2292742360c48b7875a88aee4ba641b70ade89bfa4c7224f3ad97 not found: ID does not exist"
Sep 10 12:17:19.889885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.889850    2551 scope.go:117] "RemoveContainer" containerID="70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325"
Sep 10 12:17:19.892821 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.892798    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.896495 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.896474    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.897459 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.897442    2551 scope.go:117] "RemoveContainer" containerID="70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325"
Sep 10 12:17:19.897674 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.897658    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325\": container with ID starting with 70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325 not found: ID does not exist" containerID="70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325"
Sep 10 12:17:19.897761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:19.897679    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325"} err="failed to get container status \"70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325\": rpc error: code = NotFound desc = could not find container \"70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325\": container with ID starting with 70e8f95b171d65a7d5857c377f1062939b728363e8daa382a22d96c608e44325 not found: ID does not exist"
Sep 10 12:17:19.898330 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.898312    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:19.900369 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.900353    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:19.902258 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:19.902240    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:20.037072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.037053    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:20.039779 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:20.039755    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d12fe16_85be_420d_8b9e_366874c4fd58.slice/crio-ead2374a8502033676cb798ebf39d8d8764e90cc639f215f9abdb1f23250dcdf WatchSource:0}: Error finding container ead2374a8502033676cb798ebf39d8d8764e90cc639f215f9abdb1f23250dcdf: Status 404 returned error can't find the container with id ead2374a8502033676cb798ebf39d8d8764e90cc639f215f9abdb1f23250dcdf
Sep 10 12:17:20.041691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.041673    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"]
Sep 10 12:17:20.042669 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:20.042651    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773b95ef_0eef_411d_88e9_7688a1f71055.slice/crio-c616d340ac75eeb01dcb2738c890e51b6408daa050c7303d51e2dbfe5c7f7c6f WatchSource:0}: Error finding container c616d340ac75eeb01dcb2738c890e51b6408daa050c7303d51e2dbfe5c7f7c6f: Status 404 returned error can't find the container with id c616d340ac75eeb01dcb2738c890e51b6408daa050c7303d51e2dbfe5c7f7c6f
Sep 10 12:17:20.579992 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.579959    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:17:20.583763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.583742    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.601604 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.601583    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:17:20.628361 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:20.628307    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bqvhr\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bqvhr"
Sep 10 12:17:20.631770 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:20.631751    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7xqqz\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7xqqz"
Sep 10 12:17:20.664114 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.664090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvmcz\" (UniqueName: \"kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.664196 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.664123    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.722614 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.722591    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80b16e8-65bf-4027-b195-3007e5221b21" path="/var/lib/kubelet/pods/a80b16e8-65bf-4027-b195-3007e5221b21/volumes"
Sep 10 12:17:20.722911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.722899    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b961b4e1-a554-4278-9c9f-66bac7f39925" path="/var/lib/kubelet/pods/b961b4e1-a554-4278-9c9f-66bac7f39925/volumes"
Sep 10 12:17:20.764819 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.764798    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvmcz\" (UniqueName: \"kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.764892 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.764828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.765106 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.765092    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.781213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.781185    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvmcz\" (UniqueName: \"kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rblb9\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.891357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.891290    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" event={"ID":"1d12fe16-85be-420d-8b9e-366874c4fd58","Type":"ContainerStarted","Data":"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"}
Sep 10 12:17:20.891357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.891343    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" event={"ID":"1d12fe16-85be-420d-8b9e-366874c4fd58","Type":"ContainerStarted","Data":"ead2374a8502033676cb798ebf39d8d8764e90cc639f215f9abdb1f23250dcdf"}
Sep 10 12:17:20.891553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.891404    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:20.892960 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.892939    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:20.893182 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.893163    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq" event={"ID":"773b95ef-0eef-411d-88e9-7688a1f71055","Type":"ContainerStarted","Data":"5bf9086f9e6d95c66c74388c8004efd18827ccb6f99ad0252a598483631af8cf"}
Sep 10 12:17:20.893251 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.893204    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq" event={"ID":"773b95ef-0eef-411d-88e9-7688a1f71055","Type":"ContainerStarted","Data":"c616d340ac75eeb01dcb2738c890e51b6408daa050c7303d51e2dbfe5c7f7c6f"}
Sep 10 12:17:20.893301 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.893283    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:20.957010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:20.956817    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" podStartSLOduration=1.956802178 podStartE2EDuration="1.956802178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:20.955520051 +0000 UTC m=+468.736077550" watchObservedRunningTime="2026-09-10 12:17:20.956802178 +0000 UTC m=+468.737359650"
Sep 10 12:17:21.005008 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.004958    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq" podStartSLOduration=2.004944694 podStartE2EDuration="2.004944694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:21.003654731 +0000 UTC m=+468.784212225" watchObservedRunningTime="2026-09-10 12:17:21.004944694 +0000 UTC m=+468.785502167"
Sep 10 12:17:21.190004 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.189978    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:17:21.190108 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:21.190050    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04166cbb_6445_4523_8d0d_9ce9af6fd81f.slice/crio-9bde974c353d37c708f18317e64986a37ed8efd632b36357eb5ab3d1899ca95a WatchSource:0}: Error finding container 9bde974c353d37c708f18317e64986a37ed8efd632b36357eb5ab3d1899ca95a: Status 404 returned error can't find the container with id 9bde974c353d37c708f18317e64986a37ed8efd632b36357eb5ab3d1899ca95a
Sep 10 12:17:21.900086 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.900047    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" event={"ID":"04166cbb-6445-4523-8d0d-9ce9af6fd81f","Type":"ContainerStarted","Data":"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"}
Sep 10 12:17:21.900475 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.900113    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" event={"ID":"04166cbb-6445-4523-8d0d-9ce9af6fd81f","Type":"ContainerStarted","Data":"9bde974c353d37c708f18317e64986a37ed8efd632b36357eb5ab3d1899ca95a"}
Sep 10 12:17:21.900475 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.900428    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:21.926631 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:21.926587    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" podStartSLOduration=1.9265740519999999 podStartE2EDuration="1.926574052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:21.924214103 +0000 UTC m=+469.704771585" watchObservedRunningTime="2026-09-10 12:17:21.926574052 +0000 UTC m=+469.707131526"
Sep 10 12:17:26.867881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:26.867853    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cnm8k"
Sep 10 12:17:31.901950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:31.901922    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:31.902370 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:31.901995    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jwxxq"
Sep 10 12:17:32.907361 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:32.907299    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:17:32.967515 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:32.967487    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:32.967706 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:32.967684    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" podUID="1d12fe16-85be-420d-8b9e-366874c4fd58" containerName="manager" containerID="cri-o://afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52" gracePeriod=10
Sep 10 12:17:33.210859 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.210836    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:33.352105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.352080    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pld5b\" (UniqueName: \"kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b\") pod \"1d12fe16-85be-420d-8b9e-366874c4fd58\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") "
Sep 10 12:17:33.352241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.352122    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume\") pod \"1d12fe16-85be-420d-8b9e-366874c4fd58\" (UID: \"1d12fe16-85be-420d-8b9e-366874c4fd58\") "
Sep 10 12:17:33.352531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.352510    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d12fe16-85be-420d-8b9e-366874c4fd58" (UID: "1d12fe16-85be-420d-8b9e-366874c4fd58"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:33.354283 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.354258    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b" (OuterVolumeSpecName: "kube-api-access-pld5b") pod "1d12fe16-85be-420d-8b9e-366874c4fd58" (UID: "1d12fe16-85be-420d-8b9e-366874c4fd58"). InnerVolumeSpecName "kube-api-access-pld5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:33.452723 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.452676    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pld5b\" (UniqueName: \"kubernetes.io/projected/1d12fe16-85be-420d-8b9e-366874c4fd58-kube-api-access-pld5b\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:33.452723 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.452695    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d12fe16-85be-420d-8b9e-366874c4fd58-extensions-socket-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:33.961252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.961228    2551 generic.go:356] "Generic (PLEG): container finished" podID="1d12fe16-85be-420d-8b9e-366874c4fd58" containerID="afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52" exitCode=0
Sep 10 12:17:33.961648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.961260    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"
Sep 10 12:17:33.961648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.961260    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" event={"ID":"1d12fe16-85be-420d-8b9e-366874c4fd58","Type":"ContainerDied","Data":"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"}
Sep 10 12:17:33.961648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.961299    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp" event={"ID":"1d12fe16-85be-420d-8b9e-366874c4fd58","Type":"ContainerDied","Data":"ead2374a8502033676cb798ebf39d8d8764e90cc639f215f9abdb1f23250dcdf"}
Sep 10 12:17:33.961648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.961319    2551 scope.go:117] "RemoveContainer" containerID="afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"
Sep 10 12:17:33.970201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.970184    2551 scope.go:117] "RemoveContainer" containerID="afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"
Sep 10 12:17:33.970482 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:33.970458    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52\": container with ID starting with afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52 not found: ID does not exist" containerID="afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"
Sep 10 12:17:33.970568 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.970489    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52"} err="failed to get container status \"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52\": rpc error: code = NotFound desc = could not find container \"afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52\": container with ID starting with afa3f110582845444bc099f053b06da970325aa1d4dd086d2f6b5106416a7a52 not found: ID does not exist"
Sep 10 12:17:33.984859 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.984829    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:33.990046 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:33.990027    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hs5cp"]
Sep 10 12:17:34.722587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:34.722559    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d12fe16-85be-420d-8b9e-366874c4fd58" path="/var/lib/kubelet/pods/1d12fe16-85be-420d-8b9e-366874c4fd58/volumes"
Sep 10 12:17:51.593140 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.593106    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:17:51.593529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.593513    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d12fe16-85be-420d-8b9e-366874c4fd58" containerName="manager"
Sep 10 12:17:51.593571 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.593531    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d12fe16-85be-420d-8b9e-366874c4fd58" containerName="manager"
Sep 10 12:17:51.593616 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.593607    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d12fe16-85be-420d-8b9e-366874c4fd58" containerName="manager"
Sep 10 12:17:51.597209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.597191    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.599714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.599695    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 12:17:51.599799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.599719    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pz4lb\""
Sep 10 12:17:51.606018 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.605993    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:17:51.683924 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.683895    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:17:51.687142 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.687118    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.687241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.687167    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4hjp\" (UniqueName: \"kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.788476 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.788440    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.788612 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.788507    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4hjp\" (UniqueName: \"kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.789441 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.789420    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.798202 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.798178    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4hjp\" (UniqueName: \"kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp\") pod \"limitador-limitador-8588749977-zr2rk\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") " pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:51.907702 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:51.907654    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:52.182595 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.182568    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:17:52.185224 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:52.185194    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda801e7e_437a_424f_985d_be76879a175b.slice/crio-64f51e4d031fbe27ce6c3d37d0e846f87182034ef5e501723f1383bec4e7100a WatchSource:0}: Error finding container 64f51e4d031fbe27ce6c3d37d0e846f87182034ef5e501723f1383bec4e7100a: Status 404 returned error can't find the container with id 64f51e4d031fbe27ce6c3d37d0e846f87182034ef5e501723f1383bec4e7100a
Sep 10 12:17:52.458486 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.458415    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:52.461241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.461226    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:52.463514 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.463489    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b9vcl\""
Sep 10 12:17:52.468703 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.468684    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:52.594065 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.594035    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fr9f\" (UniqueName: \"kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f\") pod \"authorino-7bcfd8f9cd-vzbsn\" (UID: \"9274219e-cf27-43b3-88e8-62797020a1ea\") " pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:52.695248 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.695220    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fr9f\" (UniqueName: \"kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f\") pod \"authorino-7bcfd8f9cd-vzbsn\" (UID: \"9274219e-cf27-43b3-88e8-62797020a1ea\") " pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:52.703714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.703691    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fr9f\" (UniqueName: \"kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f\") pod \"authorino-7bcfd8f9cd-vzbsn\" (UID: \"9274219e-cf27-43b3-88e8-62797020a1ea\") " pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:52.771504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:52.771435    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:53.046625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:53.046590    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:53.052526 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:53.052493    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" event={"ID":"da801e7e-437a-424f-985d-be76879a175b","Type":"ContainerStarted","Data":"64f51e4d031fbe27ce6c3d37d0e846f87182034ef5e501723f1383bec4e7100a"}
Sep 10 12:17:53.081036 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:17:53.081011    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9274219e_cf27_43b3_88e8_62797020a1ea.slice/crio-584a1180348eb92dece5a401bc383def5f27e446c50f347212af95b3d2b007a9 WatchSource:0}: Error finding container 584a1180348eb92dece5a401bc383def5f27e446c50f347212af95b3d2b007a9: Status 404 returned error can't find the container with id 584a1180348eb92dece5a401bc383def5f27e446c50f347212af95b3d2b007a9
Sep 10 12:17:54.059946 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:54.059907    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" event={"ID":"9274219e-cf27-43b3-88e8-62797020a1ea","Type":"ContainerStarted","Data":"584a1180348eb92dece5a401bc383def5f27e446c50f347212af95b3d2b007a9"}
Sep 10 12:17:54.061709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:54.061681    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" event={"ID":"da801e7e-437a-424f-985d-be76879a175b","Type":"ContainerStarted","Data":"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"}
Sep 10 12:17:54.061841 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:54.061798    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:17:54.081399 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:54.081339    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" podStartSLOduration=2.159696935 podStartE2EDuration="3.081322394s" totalImagesPullingTime="921.625459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:54.078413204 +0000 UTC m=+501.858970688" watchObservedRunningTime="2026-09-10 12:17:54.081322394 +0000 UTC m=+501.861879867"
Sep 10 12:17:55.067404 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:55.067298    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" event={"ID":"9274219e-cf27-43b3-88e8-62797020a1ea","Type":"ContainerStarted","Data":"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"}
Sep 10 12:17:55.084236 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:55.084192    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" podStartSLOduration=1.526976593 podStartE2EDuration="3.084179125s" totalImagesPullingTime="1.557202532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:55.082603556 +0000 UTC m=+502.863161031" watchObservedRunningTime="2026-09-10 12:17:55.084179125 +0000 UTC m=+502.864736597"
Sep 10 12:17:56.233140 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:56.233104    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:57.076243 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:57.076204    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" podUID="9274219e-cf27-43b3-88e8-62797020a1ea" containerName="authorino" containerID="cri-o://4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222" gracePeriod=30
Sep 10 12:17:57.317212 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:57.317188    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:57.428627 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:57.428604    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fr9f\" (UniqueName: \"kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f\") pod \"9274219e-cf27-43b3-88e8-62797020a1ea\" (UID: \"9274219e-cf27-43b3-88e8-62797020a1ea\") "
Sep 10 12:17:57.430459 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:57.430436    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f" (OuterVolumeSpecName: "kube-api-access-8fr9f") pod "9274219e-cf27-43b3-88e8-62797020a1ea" (UID: "9274219e-cf27-43b3-88e8-62797020a1ea"). InnerVolumeSpecName "kube-api-access-8fr9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:57.529480 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:57.529460    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fr9f\" (UniqueName: \"kubernetes.io/projected/9274219e-cf27-43b3-88e8-62797020a1ea-kube-api-access-8fr9f\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:17:58.082424 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.082396    2551 generic.go:356] "Generic (PLEG): container finished" podID="9274219e-cf27-43b3-88e8-62797020a1ea" containerID="4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222" exitCode=0
Sep 10 12:17:58.082567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.082429    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn"
Sep 10 12:17:58.082567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.082431    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" event={"ID":"9274219e-cf27-43b3-88e8-62797020a1ea","Type":"ContainerDied","Data":"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"}
Sep 10 12:17:58.082567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.082483    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-vzbsn" event={"ID":"9274219e-cf27-43b3-88e8-62797020a1ea","Type":"ContainerDied","Data":"584a1180348eb92dece5a401bc383def5f27e446c50f347212af95b3d2b007a9"}
Sep 10 12:17:58.082567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.082506    2551 scope.go:117] "RemoveContainer" containerID="4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"
Sep 10 12:17:58.090509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.090495    2551 scope.go:117] "RemoveContainer" containerID="4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"
Sep 10 12:17:58.090740 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:17:58.090723    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222\": container with ID starting with 4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222 not found: ID does not exist" containerID="4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"
Sep 10 12:17:58.090790 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.090751    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222"} err="failed to get container status \"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222\": rpc error: code = NotFound desc = could not find container \"4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222\": container with ID starting with 4916266a5e99d4a1a59e9d9a72232169c4bb71763fdef98220032266438a6222 not found: ID does not exist"
Sep 10 12:17:58.105392 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.104172    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:58.111157 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.111130    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-vzbsn"]
Sep 10 12:17:58.722149 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:17:58.722117    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9274219e-cf27-43b3-88e8-62797020a1ea" path="/var/lib/kubelet/pods/9274219e-cf27-43b3-88e8-62797020a1ea/volumes"
Sep 10 12:18:05.068394 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:05.068344    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:18:06.234289 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.234260    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:18:06.234690 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.234451    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" podUID="da801e7e-437a-424f-985d-be76879a175b" containerName="limitador" containerID="cri-o://c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840" gracePeriod=30
Sep 10 12:18:06.766525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.766501    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:18:06.894929 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.894908    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4hjp\" (UniqueName: \"kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp\") pod \"da801e7e-437a-424f-985d-be76879a175b\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") "
Sep 10 12:18:06.895053 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.894941    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file\") pod \"da801e7e-437a-424f-985d-be76879a175b\" (UID: \"da801e7e-437a-424f-985d-be76879a175b\") "
Sep 10 12:18:06.895316 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.895297    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file" (OuterVolumeSpecName: "config-file") pod "da801e7e-437a-424f-985d-be76879a175b" (UID: "da801e7e-437a-424f-985d-be76879a175b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:18:06.896813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.896793    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp" (OuterVolumeSpecName: "kube-api-access-d4hjp") pod "da801e7e-437a-424f-985d-be76879a175b" (UID: "da801e7e-437a-424f-985d-be76879a175b"). InnerVolumeSpecName "kube-api-access-d4hjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:18:06.996034 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.996015    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4hjp\" (UniqueName: \"kubernetes.io/projected/da801e7e-437a-424f-985d-be76879a175b-kube-api-access-d4hjp\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:06.996117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:06.996037    2551 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da801e7e-437a-424f-985d-be76879a175b-config-file\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:07.126171 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.126143    2551 generic.go:356] "Generic (PLEG): container finished" podID="da801e7e-437a-424f-985d-be76879a175b" containerID="c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840" exitCode=0
Sep 10 12:18:07.126257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.126180    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk"
Sep 10 12:18:07.126257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.126195    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" event={"ID":"da801e7e-437a-424f-985d-be76879a175b","Type":"ContainerDied","Data":"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"}
Sep 10 12:18:07.126257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.126239    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zr2rk" event={"ID":"da801e7e-437a-424f-985d-be76879a175b","Type":"ContainerDied","Data":"64f51e4d031fbe27ce6c3d37d0e846f87182034ef5e501723f1383bec4e7100a"}
Sep 10 12:18:07.126356 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.126262    2551 scope.go:117] "RemoveContainer" containerID="c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"
Sep 10 12:18:07.134559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.134530    2551 scope.go:117] "RemoveContainer" containerID="c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"
Sep 10 12:18:07.134795 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:18:07.134776    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840\": container with ID starting with c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840 not found: ID does not exist" containerID="c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"
Sep 10 12:18:07.134841 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.134804    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840"} err="failed to get container status \"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840\": rpc error: code = NotFound desc = could not find container \"c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840\": container with ID starting with c069350476140c004a2b1089cb743a3b254cc48c174eef800aff18bdb683f840 not found: ID does not exist"
Sep 10 12:18:07.146987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.146942    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:18:07.150206 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:07.150186    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zr2rk"]
Sep 10 12:18:08.042523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042494    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"]
Sep 10 12:18:08.042873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042806    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da801e7e-437a-424f-985d-be76879a175b" containerName="limitador"
Sep 10 12:18:08.042873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042817    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="da801e7e-437a-424f-985d-be76879a175b" containerName="limitador"
Sep 10 12:18:08.042873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042838    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9274219e-cf27-43b3-88e8-62797020a1ea" containerName="authorino"
Sep 10 12:18:08.042873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042843    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9274219e-cf27-43b3-88e8-62797020a1ea" containerName="authorino"
Sep 10 12:18:08.043003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042908    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="da801e7e-437a-424f-985d-be76879a175b" containerName="limitador"
Sep 10 12:18:08.043003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.042920    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9274219e-cf27-43b3-88e8-62797020a1ea" containerName="authorino"
Sep 10 12:18:08.045812 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.045791    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.048272 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.048250    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 12:18:08.048400 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.048250    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 12:18:08.049267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.049253    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 12:18:08.049340 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.049256    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-7vct5\""
Sep 10 12:18:08.054204 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.054182    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"]
Sep 10 12:18:08.102860 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.102840    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7896\" (UniqueName: \"kubernetes.io/projected/5dd7764b-e247-4a5c-b862-b2c0b093e348-kube-api-access-x7896\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.102970 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.102893    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5dd7764b-e247-4a5c-b862-b2c0b093e348-data\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.203460 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.203438    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7896\" (UniqueName: \"kubernetes.io/projected/5dd7764b-e247-4a5c-b862-b2c0b093e348-kube-api-access-x7896\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.203559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.203500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5dd7764b-e247-4a5c-b862-b2c0b093e348-data\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.203779 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.203763    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5dd7764b-e247-4a5c-b862-b2c0b093e348-data\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.212766 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.212747    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7896\" (UniqueName: \"kubernetes.io/projected/5dd7764b-e247-4a5c-b862-b2c0b093e348-kube-api-access-x7896\") pod \"postgres-69fb4594cf-7ng6x\" (UID: \"5dd7764b-e247-4a5c-b862-b2c0b093e348\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.357857 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.357841    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:08.626018 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.625957    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"]
Sep 10 12:18:08.629024 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:18:08.628996    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd7764b_e247_4a5c_b862_b2c0b093e348.slice/crio-a25e6986d124c7161597f29013ea435ee37b829b3b707fd469dfdd8f7ccf4231 WatchSource:0}: Error finding container a25e6986d124c7161597f29013ea435ee37b829b3b707fd469dfdd8f7ccf4231: Status 404 returned error can't find the container with id a25e6986d124c7161597f29013ea435ee37b829b3b707fd469dfdd8f7ccf4231
Sep 10 12:18:08.722900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:08.722875    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da801e7e-437a-424f-985d-be76879a175b" path="/var/lib/kubelet/pods/da801e7e-437a-424f-985d-be76879a175b/volumes"
Sep 10 12:18:09.139860 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:09.139822    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x" event={"ID":"5dd7764b-e247-4a5c-b862-b2c0b093e348","Type":"ContainerStarted","Data":"a25e6986d124c7161597f29013ea435ee37b829b3b707fd469dfdd8f7ccf4231"}
Sep 10 12:18:13.164667 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:13.164636    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x" event={"ID":"5dd7764b-e247-4a5c-b862-b2c0b093e348","Type":"ContainerStarted","Data":"8528cfafa007773ca7f9891a677bf37406eb37f5ac0a26894ee6731aae93407d"}
Sep 10 12:18:13.165024 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:13.164770    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:13.182267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:13.182223    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x" podStartSLOduration=0.827881778 podStartE2EDuration="5.182210339s" totalImagesPullingTime="4.354328561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.180644676 +0000 UTC m=+520.961202179" watchObservedRunningTime="2026-09-10 12:18:13.182210339 +0000 UTC m=+520.962767812"
Sep 10 12:18:19.196521 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.196492    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7ng6x"
Sep 10 12:18:19.728725 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.728697    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-wqnf8"]
Sep 10 12:18:19.734841 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.734820    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:19.736425 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.736399    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-wqnf8"]
Sep 10 12:18:19.737228 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.737207    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b9vcl\""
Sep 10 12:18:19.802635 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.802610    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xw2t\" (UniqueName: \"kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t\") pod \"authorino-557f876f69-wqnf8\" (UID: \"a6b8ca65-8600-405c-8d0a-fc805031395b\") " pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:19.864962 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.864936    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wqnf8"]
Sep 10 12:18:19.865166 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:18:19.865147    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5xw2t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-wqnf8" podUID="a6b8ca65-8600-405c-8d0a-fc805031395b"
Sep 10 12:18:19.897993 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.897971    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qq2rg"]
Sep 10 12:18:19.902006 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.901990    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:19.903030 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.902980    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xw2t\" (UniqueName: \"kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t\") pod \"authorino-557f876f69-wqnf8\" (UID: \"a6b8ca65-8600-405c-8d0a-fc805031395b\") " pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:19.904649 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.904632    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 12:18:19.909813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.909792    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qq2rg"]
Sep 10 12:18:19.917905 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.917886    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xw2t\" (UniqueName: \"kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t\") pod \"authorino-557f876f69-wqnf8\" (UID: \"a6b8ca65-8600-405c-8d0a-fc805031395b\") " pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:19.977920 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:19.977895    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qq2rg"]
Sep 10 12:18:19.978108 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:18:19.978091    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tmkm4 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-tmkm4 tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-qq2rg" podUID="722c919a-dd9e-4a9d-9644-304cf42f60f6"
Sep 10 12:18:20.003665 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.003615    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.003665 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.003642    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmkm4\" (UniqueName: \"kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.104481 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.104456    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.104561 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.104485    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmkm4\" (UniqueName: \"kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.106762 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.106743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.113472 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.113454    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmkm4\" (UniqueName: \"kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4\") pod \"authorino-6d4dfc649f-qq2rg\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") " pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.196075 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.196048    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.196166 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.196054    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:20.203536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.203515    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:20.206292 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.206278    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:20.305493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.305447    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xw2t\" (UniqueName: \"kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t\") pod \"a6b8ca65-8600-405c-8d0a-fc805031395b\" (UID: \"a6b8ca65-8600-405c-8d0a-fc805031395b\") "
Sep 10 12:18:20.305493 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.305482    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmkm4\" (UniqueName: \"kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4\") pod \"722c919a-dd9e-4a9d-9644-304cf42f60f6\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") "
Sep 10 12:18:20.305611 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.305542    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert\") pod \"722c919a-dd9e-4a9d-9644-304cf42f60f6\" (UID: \"722c919a-dd9e-4a9d-9644-304cf42f60f6\") "
Sep 10 12:18:20.307285 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.307259    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t" (OuterVolumeSpecName: "kube-api-access-5xw2t") pod "a6b8ca65-8600-405c-8d0a-fc805031395b" (UID: "a6b8ca65-8600-405c-8d0a-fc805031395b"). InnerVolumeSpecName "kube-api-access-5xw2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:18:20.307536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.307507    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "722c919a-dd9e-4a9d-9644-304cf42f60f6" (UID: "722c919a-dd9e-4a9d-9644-304cf42f60f6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:18:20.307536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.307513    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4" (OuterVolumeSpecName: "kube-api-access-tmkm4") pod "722c919a-dd9e-4a9d-9644-304cf42f60f6" (UID: "722c919a-dd9e-4a9d-9644-304cf42f60f6"). InnerVolumeSpecName "kube-api-access-tmkm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:18:20.406949 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.406929    2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/722c919a-dd9e-4a9d-9644-304cf42f60f6-tls-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:20.407028 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.406949    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xw2t\" (UniqueName: \"kubernetes.io/projected/a6b8ca65-8600-405c-8d0a-fc805031395b-kube-api-access-5xw2t\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:20.407028 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:20.406959    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmkm4\" (UniqueName: \"kubernetes.io/projected/722c919a-dd9e-4a9d-9644-304cf42f60f6-kube-api-access-tmkm4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:21.200648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.200619    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wqnf8"
Sep 10 12:18:21.200748 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.200628    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qq2rg"
Sep 10 12:18:21.231016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.230986    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qq2rg"]
Sep 10 12:18:21.234758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.234734    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qq2rg"]
Sep 10 12:18:21.256193 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.256168    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wqnf8"]
Sep 10 12:18:21.259744 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:21.259721    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-wqnf8"]
Sep 10 12:18:22.722341 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:22.722310    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="722c919a-dd9e-4a9d-9644-304cf42f60f6" path="/var/lib/kubelet/pods/722c919a-dd9e-4a9d-9644-304cf42f60f6/volumes"
Sep 10 12:18:22.722707 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:22.722547    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b8ca65-8600-405c-8d0a-fc805031395b" path="/var/lib/kubelet/pods/a6b8ca65-8600-405c-8d0a-fc805031395b/volumes"
Sep 10 12:18:26.037347 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.037269    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:26.064472 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.064445    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:26.064631 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.064526    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.067357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.067332    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:18:26.067504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.067332    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hmjln\""
Sep 10 12:18:26.067598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.067339    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:18:26.068261 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.068092    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:18:26.151000 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.150979    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.151105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.151070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.151105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.151100    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjtk\" (UniqueName: \"kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.217124 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.217099    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:18:26.220606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.220589    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.231311 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.231288    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:18:26.251517 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251492    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.251617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251532    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjtk\" (UniqueName: \"kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.251617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.251617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251583    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.251773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251636    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xgl9\" (UniqueName: \"kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.251773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.251659    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.254131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.254105    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.254257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.254239    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.260444 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.260427    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjtk\" (UniqueName: \"kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk\") pod \"maas-controller-698886f5c9-lzs57\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") " pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.352257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.352235    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.352357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.352283    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xgl9\" (UniqueName: \"kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.352357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.352302    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.354660 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.354642    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.354742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.354695    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.362950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.362929    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xgl9\" (UniqueName: \"kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9\") pod \"maas-controller-d6f6f5d54-qk9x9\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") " pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.377641 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.377620    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:26.529894 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.529866    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:26.647060 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.647031    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:26.648867 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:18:26.648847    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod669bf143_984b_40fa_be71_631e98b43b77.slice/crio-9b2c4cd4b77f620c00f099c14dccfe57cf9b389bea2d1673530febad9f3e826d WatchSource:0}: Error finding container 9b2c4cd4b77f620c00f099c14dccfe57cf9b389bea2d1673530febad9f3e826d: Status 404 returned error can't find the container with id 9b2c4cd4b77f620c00f099c14dccfe57cf9b389bea2d1673530febad9f3e826d
Sep 10 12:18:26.801518 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:26.801498    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:18:26.803405 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:18:26.803360    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9faa6201_2130_402d_9408_acb9b353f5ba.slice/crio-ce52601d4e6f1b042c331400127c704405d55c2372dc38fb3f606d215c04e1ba WatchSource:0}: Error finding container ce52601d4e6f1b042c331400127c704405d55c2372dc38fb3f606d215c04e1ba: Status 404 returned error can't find the container with id ce52601d4e6f1b042c331400127c704405d55c2372dc38fb3f606d215c04e1ba
Sep 10 12:18:27.230531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:27.230501    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698886f5c9-lzs57" event={"ID":"669bf143-984b-40fa-be71-631e98b43b77","Type":"ContainerStarted","Data":"9b2c4cd4b77f620c00f099c14dccfe57cf9b389bea2d1673530febad9f3e826d"}
Sep 10 12:18:27.231664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:27.231640    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" event={"ID":"9faa6201-2130-402d-9408-acb9b353f5ba","Type":"ContainerStarted","Data":"ce52601d4e6f1b042c331400127c704405d55c2372dc38fb3f606d215c04e1ba"}
Sep 10 12:18:30.249900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.249856    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" event={"ID":"9faa6201-2130-402d-9408-acb9b353f5ba","Type":"ContainerStarted","Data":"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"}
Sep 10 12:18:30.250297 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.249926    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:30.251483 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.251457    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698886f5c9-lzs57" event={"ID":"669bf143-984b-40fa-be71-631e98b43b77","Type":"ContainerStarted","Data":"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"}
Sep 10 12:18:30.251609 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.251592    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:30.274400 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.274347    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" podStartSLOduration=1.847472421 podStartE2EDuration="4.274334872s" totalImagesPullingTime="2.426862451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:30.272764567 +0000 UTC m=+538.053322039" watchObservedRunningTime="2026-09-10 12:18:30.274334872 +0000 UTC m=+538.054892386"
Sep 10 12:18:30.301908 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:30.301861    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-698886f5c9-lzs57" podStartSLOduration=1.72736534 podStartE2EDuration="4.301844733s" totalImagesPullingTime="2.574479393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:30.299818024 +0000 UTC m=+538.080375524" watchObservedRunningTime="2026-09-10 12:18:30.301844733 +0000 UTC m=+538.082402206"
Sep 10 12:18:41.259762 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.259729    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:41.260140 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.259800    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:18:41.342937 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.342900    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:41.343180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.343138    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-698886f5c9-lzs57" podUID="669bf143-984b-40fa-be71-631e98b43b77" containerName="manager" containerID="cri-o://beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b" gracePeriod=30
Sep 10 12:18:41.581938 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.581913    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:41.663465 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.663436    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert\") pod \"669bf143-984b-40fa-be71-631e98b43b77\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") "
Sep 10 12:18:41.663612 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.663519    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkjtk\" (UniqueName: \"kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk\") pod \"669bf143-984b-40fa-be71-631e98b43b77\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") "
Sep 10 12:18:41.663612 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.663566    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs\") pod \"669bf143-984b-40fa-be71-631e98b43b77\" (UID: \"669bf143-984b-40fa-be71-631e98b43b77\") "
Sep 10 12:18:41.665646 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.665619    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "669bf143-984b-40fa-be71-631e98b43b77" (UID: "669bf143-984b-40fa-be71-631e98b43b77"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:18:41.665646 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.665633    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "669bf143-984b-40fa-be71-631e98b43b77" (UID: "669bf143-984b-40fa-be71-631e98b43b77"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:18:41.665836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.665671    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk" (OuterVolumeSpecName: "kube-api-access-nkjtk") pod "669bf143-984b-40fa-be71-631e98b43b77" (UID: "669bf143-984b-40fa-be71-631e98b43b77"). InnerVolumeSpecName "kube-api-access-nkjtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:18:41.765010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.764981    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:41.765010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.765006    2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/669bf143-984b-40fa-be71-631e98b43b77-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:41.765152 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:41.765019    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkjtk\" (UniqueName: \"kubernetes.io/projected/669bf143-984b-40fa-be71-631e98b43b77-kube-api-access-nkjtk\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:18:42.309398 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.309346    2551 generic.go:356] "Generic (PLEG): container finished" podID="669bf143-984b-40fa-be71-631e98b43b77" containerID="beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b" exitCode=0
Sep 10 12:18:42.309836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.309411    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698886f5c9-lzs57"
Sep 10 12:18:42.309836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.309413    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698886f5c9-lzs57" event={"ID":"669bf143-984b-40fa-be71-631e98b43b77","Type":"ContainerDied","Data":"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"}
Sep 10 12:18:42.309836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.309471    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698886f5c9-lzs57" event={"ID":"669bf143-984b-40fa-be71-631e98b43b77","Type":"ContainerDied","Data":"9b2c4cd4b77f620c00f099c14dccfe57cf9b389bea2d1673530febad9f3e826d"}
Sep 10 12:18:42.309836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.309492    2551 scope.go:117] "RemoveContainer" containerID="beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"
Sep 10 12:18:42.318223 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.318210    2551 scope.go:117] "RemoveContainer" containerID="beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"
Sep 10 12:18:42.318506 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:18:42.318488    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b\": container with ID starting with beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b not found: ID does not exist" containerID="beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"
Sep 10 12:18:42.318563 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.318516    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b"} err="failed to get container status \"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b\": rpc error: code = NotFound desc = could not find container \"beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b\": container with ID starting with beb80b9435fdc0d9b49676f49564ac44519bc54e8eaae4aa86a1f46c98fee24b not found: ID does not exist"
Sep 10 12:18:42.331022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.331002    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:42.337361 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.337338    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-698886f5c9-lzs57"]
Sep 10 12:18:42.722633 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:18:42.722608    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="669bf143-984b-40fa-be71-631e98b43b77" path="/var/lib/kubelet/pods/669bf143-984b-40fa-be71-631e98b43b77/volumes"
Sep 10 12:19:00.967587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.967528    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:19:00.968119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.968100    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="669bf143-984b-40fa-be71-631e98b43b77" containerName="manager"
Sep 10 12:19:00.968196 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.968124    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="669bf143-984b-40fa-be71-631e98b43b77" containerName="manager"
Sep 10 12:19:00.968247 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.968233    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="669bf143-984b-40fa-be71-631e98b43b77" containerName="manager"
Sep 10 12:19:00.971281 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.971261    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:00.973780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.973759    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 12:19:00.974335 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.974318    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 12:19:00.982911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:00.982891    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:19:01.102568 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.102544    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.102698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.102583    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.102698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.102624    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.102698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.102684    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g6nr\" (UniqueName: \"kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.203961 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.203934    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.204110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.203975    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.204110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.204010    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.204110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.204037    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g6nr\" (UniqueName: \"kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.204110 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:19:01.204094    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 12:19:01.204317 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:19:01.204171    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs podName:f1da36ce-144f-4e72-a4ed-06724ae8bd7e nodeName:}" failed. No retries permitted until 2026-09-10 12:19:01.704153884 +0000 UTC m=+569.484711339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs") pod "maas-api-59cc544454-6kg78" (UID: "f1da36ce-144f-4e72-a4ed-06724ae8bd7e") : secret "maas-api-metrics-tls" not found
Sep 10 12:19:01.204771 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.204754    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.206385 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.206345    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.212778 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.212756    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g6nr\" (UniqueName: \"kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.513604 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.513571    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"]
Sep 10 12:19:01.517101 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.517085    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.519415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.519397    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vxsrx\""
Sep 10 12:19:01.519785 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.519770    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 12:19:01.532014 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.531995    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"]
Sep 10 12:19:01.542572 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.542550    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"]
Sep 10 12:19:01.546343 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.546320    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.556640 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.556615    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"]
Sep 10 12:19:01.607571 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.607544    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/016c941a-48e6-4de3-bead-b8e8f9fe7678-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.607676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.607575    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rvxz\" (UniqueName: \"kubernetes.io/projected/016c941a-48e6-4de3-bead-b8e8f9fe7678-kube-api-access-7rvxz\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.708106 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708082    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.708219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708122    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fc82292-8457-437d-89d5-6b0cc827bc4f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.708219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708171    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52m8\" (UniqueName: \"kubernetes.io/projected/8fc82292-8457-437d-89d5-6b0cc827bc4f-kube-api-access-b52m8\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.708219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708211    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/016c941a-48e6-4de3-bead-b8e8f9fe7678-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.708414 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708231    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rvxz\" (UniqueName: \"kubernetes.io/projected/016c941a-48e6-4de3-bead-b8e8f9fe7678-kube-api-access-7rvxz\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.708979 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.708961    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/016c941a-48e6-4de3-bead-b8e8f9fe7678-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.710415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.710367    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") pod \"maas-api-59cc544454-6kg78\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") " pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:01.730117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.730091    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rvxz\" (UniqueName: \"kubernetes.io/projected/016c941a-48e6-4de3-bead-b8e8f9fe7678-kube-api-access-7rvxz\") pod \"payload-pre-processing-59fbc47bf8-ckqsj\" (UID: \"016c941a-48e6-4de3-bead-b8e8f9fe7678\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.809109 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.809038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fc82292-8457-437d-89d5-6b0cc827bc4f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.809275 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.809139    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52m8\" (UniqueName: \"kubernetes.io/projected/8fc82292-8457-437d-89d5-6b0cc827bc4f-kube-api-access-b52m8\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.809777 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.809756    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fc82292-8457-437d-89d5-6b0cc827bc4f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.817781 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.817760    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52m8\" (UniqueName: \"kubernetes.io/projected/8fc82292-8457-437d-89d5-6b0cc827bc4f-kube-api-access-b52m8\") pod \"payload-processing-5b54b7fbb7-zpkr4\" (UID: \"8fc82292-8457-437d-89d5-6b0cc827bc4f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.827567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.827547    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:01.858459 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.858437    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:01.882813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:01.882698    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:02.096553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.096530    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"]
Sep 10 12:19:02.098107 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:19:02.098085    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod016c941a_48e6_4de3_bead_b8e8f9fe7678.slice/crio-fb097a609aa0a2a4230dd85c44383e3545c5e486ef7138afb1da7ff1b1202a3a WatchSource:0}: Error finding container fb097a609aa0a2a4230dd85c44383e3545c5e486ef7138afb1da7ff1b1202a3a: Status 404 returned error can't find the container with id fb097a609aa0a2a4230dd85c44383e3545c5e486ef7138afb1da7ff1b1202a3a
Sep 10 12:19:02.145851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.145829    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"]
Sep 10 12:19:02.147857 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:19:02.147832    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc82292_8457_437d_89d5_6b0cc827bc4f.slice/crio-5d5981f9bab20087cdca5e97613da436791cbb2f5f0e3f5602e6cd5bac248a4b WatchSource:0}: Error finding container 5d5981f9bab20087cdca5e97613da436791cbb2f5f0e3f5602e6cd5bac248a4b: Status 404 returned error can't find the container with id 5d5981f9bab20087cdca5e97613da436791cbb2f5f0e3f5602e6cd5bac248a4b
Sep 10 12:19:02.172805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.172773    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:19:02.174645 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:19:02.174624    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1da36ce_144f_4e72_a4ed_06724ae8bd7e.slice/crio-d7131222d5fd6391c1f2b2c32ca034b1a2ab22f386a10b3860928085f91c8887 WatchSource:0}: Error finding container d7131222d5fd6391c1f2b2c32ca034b1a2ab22f386a10b3860928085f91c8887: Status 404 returned error can't find the container with id d7131222d5fd6391c1f2b2c32ca034b1a2ab22f386a10b3860928085f91c8887
Sep 10 12:19:02.411016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.410940    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4" event={"ID":"8fc82292-8457-437d-89d5-6b0cc827bc4f","Type":"ContainerStarted","Data":"5d5981f9bab20087cdca5e97613da436791cbb2f5f0e3f5602e6cd5bac248a4b"}
Sep 10 12:19:02.412346 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.412323    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj" event={"ID":"016c941a-48e6-4de3-bead-b8e8f9fe7678","Type":"ContainerStarted","Data":"fb097a609aa0a2a4230dd85c44383e3545c5e486ef7138afb1da7ff1b1202a3a"}
Sep 10 12:19:02.413669 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:02.413650    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" event={"ID":"f1da36ce-144f-4e72-a4ed-06724ae8bd7e","Type":"ContainerStarted","Data":"d7131222d5fd6391c1f2b2c32ca034b1a2ab22f386a10b3860928085f91c8887"}
Sep 10 12:19:05.432243 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.432206    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4" event={"ID":"8fc82292-8457-437d-89d5-6b0cc827bc4f","Type":"ContainerStarted","Data":"6eaf27e2cc2f9f6c3e85b2b5a0d89a41e4301d6bd6f1bcfd0239fe21056c73f8"}
Sep 10 12:19:05.432712 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.432328    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:05.434230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.434175    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj" event={"ID":"016c941a-48e6-4de3-bead-b8e8f9fe7678","Type":"ContainerStarted","Data":"2d9cba6e0c4ddd4518c735fd6a0f0d827e614218c11decd1aff258da2e980940"}
Sep 10 12:19:05.434458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.434310    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:05.450727 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.450683    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4" podStartSLOduration=2.238537715 podStartE2EDuration="4.450666816s" totalImagesPullingTime="2.212129101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:05.448676977 +0000 UTC m=+573.229234474" watchObservedRunningTime="2026-09-10 12:19:05.450666816 +0000 UTC m=+573.231224289"
Sep 10 12:19:05.465344 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:05.465302    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj" podStartSLOduration=2.209434956 podStartE2EDuration="4.465289919s" totalImagesPullingTime="2.255854963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:05.463895232 +0000 UTC m=+573.244452730" watchObservedRunningTime="2026-09-10 12:19:05.465289919 +0000 UTC m=+573.245847391"
Sep 10 12:19:06.441163 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:06.441126    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" event={"ID":"f1da36ce-144f-4e72-a4ed-06724ae8bd7e","Type":"ContainerStarted","Data":"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"}
Sep 10 12:19:06.441583 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:06.441398    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:06.461295 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:06.461253    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" podStartSLOduration=2.929008036 podStartE2EDuration="6.461241567s" totalImagesPullingTime="3.532233531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:06.459725761 +0000 UTC m=+574.240283245" watchObservedRunningTime="2026-09-10 12:19:06.461241567 +0000 UTC m=+574.241799040"
Sep 10 12:19:12.451079 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:12.451052    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:19:16.443307 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:16.443276    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-zpkr4"
Sep 10 12:19:16.443763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:16.443338    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ckqsj"
Sep 10 12:19:32.628352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:32.628328    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:19:32.628860 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:32.628754    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:19:32.634606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:32.634586    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:19:32.634727 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:32.634610    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:19:40.954598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.954565    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"]
Sep 10 12:19:40.959763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.959743    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:40.962108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.962085    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 12:19:40.963185 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.963166    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 12:19:40.963304 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.963167    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 12:19:40.963304 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.963272    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x8ww2\""
Sep 10 12:19:40.970733 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:40.970710    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"]
Sep 10 12:19:41.010241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010217    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.010349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010259    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.010349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010296    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.010502 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010344    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl524\" (UniqueName: \"kubernetes.io/projected/efaf8909-40e0-4329-ab3c-eca26be6a5b8-kube-api-access-pl524\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.010502 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010415    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.010502 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.010489    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111097    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111126    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl524\" (UniqueName: \"kubernetes.io/projected/efaf8909-40e0-4329-ab3c-eca26be6a5b8-kube-api-access-pl524\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111213 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111145    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111271    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111308    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111465 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111342    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111550    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111619 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111584    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.111734 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.111713    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.113250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.113231    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaf8909-40e0-4329-ab3c-eca26be6a5b8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.113565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.113548    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaf8909-40e0-4329-ab3c-eca26be6a5b8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.119339 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.119320    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl524\" (UniqueName: \"kubernetes.io/projected/efaf8909-40e0-4329-ab3c-eca26be6a5b8-kube-api-access-pl524\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5\" (UID: \"efaf8909-40e0-4329-ab3c-eca26be6a5b8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.272519 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.272467    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:41.541602 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.541578    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"]
Sep 10 12:19:41.543392 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:19:41.543341    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefaf8909_40e0_4329_ab3c_eca26be6a5b8.slice/crio-1c34ceaf44e07fb4825c21d964f43ba9e4cf05f1bec94d7b96fa630de2fbcaa4 WatchSource:0}: Error finding container 1c34ceaf44e07fb4825c21d964f43ba9e4cf05f1bec94d7b96fa630de2fbcaa4: Status 404 returned error can't find the container with id 1c34ceaf44e07fb4825c21d964f43ba9e4cf05f1bec94d7b96fa630de2fbcaa4
Sep 10 12:19:41.615052 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:41.615023    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5" event={"ID":"efaf8909-40e0-4329-ab3c-eca26be6a5b8","Type":"ContainerStarted","Data":"1c34ceaf44e07fb4825c21d964f43ba9e4cf05f1bec94d7b96fa630de2fbcaa4"}
Sep 10 12:19:42.621389 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:42.621338    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5" event={"ID":"efaf8909-40e0-4329-ab3c-eca26be6a5b8","Type":"ContainerStarted","Data":"888812bb4a0f011ce2e58301c3d4ccd8af251cb9dfc9ed55ba0734ba34e649ab"}
Sep 10 12:19:42.640664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:42.640617    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5" podStartSLOduration=1.735258227 podStartE2EDuration="2.640604243s" totalImagesPullingTime="905.346016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:42.638713301 +0000 UTC m=+610.419270788" watchObservedRunningTime="2026-09-10 12:19:42.640604243 +0000 UTC m=+610.421161715"
Sep 10 12:19:51.272697 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:51.272662    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:51.273138 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:51.272723    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:51.284675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:51.284654    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:19:51.685180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:19:51.685153    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5"
Sep 10 12:20:17.273795 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.273762    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"]
Sep 10 12:20:17.279956 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.279935    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.282996 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.282977    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 12:20:17.288630 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.288609    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"]
Sep 10 12:20:17.377508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377477    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e346e75-bad3-4e80-be42-5b47985c1de7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.377643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377514    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.377643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377537    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.377643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377615    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.377760 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377684    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.377760 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.377719    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zk9f\" (UniqueName: \"kubernetes.io/projected/1e346e75-bad3-4e80-be42-5b47985c1de7-kube-api-access-7zk9f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478331 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478299    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478364    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478425    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zk9f\" (UniqueName: \"kubernetes.io/projected/1e346e75-bad3-4e80-be42-5b47985c1de7-kube-api-access-7zk9f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478457    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e346e75-bad3-4e80-be42-5b47985c1de7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478683 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478662    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478810 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478723    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.478810 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.478742    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.479043 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.479020    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.479152 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.479091    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.480739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.480717    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e346e75-bad3-4e80-be42-5b47985c1de7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.480845 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.480773    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e346e75-bad3-4e80-be42-5b47985c1de7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.486565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.486545    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zk9f\" (UniqueName: \"kubernetes.io/projected/1e346e75-bad3-4e80-be42-5b47985c1de7-kube-api-access-7zk9f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wpgf6\" (UID: \"1e346e75-bad3-4e80-be42-5b47985c1de7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.590585 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.590560    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:17.862011 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.861985    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"]
Sep 10 12:20:17.863313 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:20:17.863286    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e346e75_bad3_4e80_be42_5b47985c1de7.slice/crio-b899a2118e399010e8e016aaea30342e135a1cdbba0274706f73e4ada7892c72 WatchSource:0}: Error finding container b899a2118e399010e8e016aaea30342e135a1cdbba0274706f73e4ada7892c72: Status 404 returned error can't find the container with id b899a2118e399010e8e016aaea30342e135a1cdbba0274706f73e4ada7892c72
Sep 10 12:20:17.864941 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:17.864925    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:20:18.802758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:18.802722    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6" event={"ID":"1e346e75-bad3-4e80-be42-5b47985c1de7","Type":"ContainerStarted","Data":"0abf3fe19c66d66e328c59f3adba4da72bd5bb0894214f6ccaf06d5e118b7605"}
Sep 10 12:20:18.803108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:18.802800    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6" event={"ID":"1e346e75-bad3-4e80-be42-5b47985c1de7","Type":"ContainerStarted","Data":"b899a2118e399010e8e016aaea30342e135a1cdbba0274706f73e4ada7892c72"}
Sep 10 12:20:18.825894 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:18.825847    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6" podStartSLOduration=1.612321992 podStartE2EDuration="1.82583402s" totalImagesPullingTime="213.512028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:18.823625724 +0000 UTC m=+646.604183197" watchObservedRunningTime="2026-09-10 12:20:18.82583402 +0000 UTC m=+646.606391493"
Sep 10 12:20:27.591243 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:27.591207    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:27.591637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:27.591264    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:27.603361 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:27.603340    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:20:27.852974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:20:27.852906    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wpgf6"
Sep 10 12:21:53.440712 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.440639    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b694d445c-g8kmn"]
Sep 10 12:21:53.444241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.444223    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.447824 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.447800    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b9vcl\""
Sep 10 12:21:53.447926 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.447825    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 12:21:53.457778 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.457759    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b694d445c-g8kmn"]
Sep 10 12:21:53.603990 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.603967    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9g6\" (UniqueName: \"kubernetes.io/projected/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-kube-api-access-wn9g6\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.604114 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.604080    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-tls-cert\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.705446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.705384    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-tls-cert\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.705446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.705420    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9g6\" (UniqueName: \"kubernetes.io/projected/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-kube-api-access-wn9g6\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.707721 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.707707    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-tls-cert\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.714915 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.714890    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9g6\" (UniqueName: \"kubernetes.io/projected/7ff6c168-88f2-49a5-b3ee-0ab8253f0a00-kube-api-access-wn9g6\") pod \"authorino-6b694d445c-g8kmn\" (UID: \"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00\") " pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:53.753454 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:53.753432    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b694d445c-g8kmn"
Sep 10 12:21:54.021478 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:54.021453    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b694d445c-g8kmn"]
Sep 10 12:21:54.023347 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:21:54.023321    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ff6c168_88f2_49a5_b3ee_0ab8253f0a00.slice/crio-d848ca72f39f3d6196c021b8d9c6ce2d88a6b5cd13e8611564ba208f179c36b1 WatchSource:0}: Error finding container d848ca72f39f3d6196c021b8d9c6ce2d88a6b5cd13e8611564ba208f179c36b1: Status 404 returned error can't find the container with id d848ca72f39f3d6196c021b8d9c6ce2d88a6b5cd13e8611564ba208f179c36b1
Sep 10 12:21:54.247902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:54.247874    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b694d445c-g8kmn" event={"ID":"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00","Type":"ContainerStarted","Data":"d848ca72f39f3d6196c021b8d9c6ce2d88a6b5cd13e8611564ba208f179c36b1"}
Sep 10 12:21:55.253802 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:55.253705    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b694d445c-g8kmn" event={"ID":"7ff6c168-88f2-49a5-b3ee-0ab8253f0a00","Type":"ContainerStarted","Data":"3cad2ee6f623862832cb048fd00fe318a5db659054857af5c82e23b3b19dd569"}
Sep 10 12:21:55.279130 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:21:55.279058    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b694d445c-g8kmn" podStartSLOduration=1.451760291 podStartE2EDuration="2.279036891s" totalImagesPullingTime="827.2766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:21:55.27710168 +0000 UTC m=+743.057659151" watchObservedRunningTime="2026-09-10 12:21:55.279036891 +0000 UTC m=+743.059594422"
Sep 10 12:22:17.055110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.055075    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:22:17.058963 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.058941    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.063876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.063854    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-89jb9\""
Sep 10 12:22:17.069744 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.069725    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7576a1-serving-cert\""
Sep 10 12:22:17.082395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.082358    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:22:17.105757 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.105736    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.105873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.105769    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82q5f\" (UniqueName: \"kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.105873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.105800    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.105983 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.105902    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.206434 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.206406    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.206569 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.206444    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82q5f\" (UniqueName: \"kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.206569 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.206468    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.206569 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.206503    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.206719 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:17.206619    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-serving-cert: secret "maas-api-e2e-inf-a-7576a1-serving-cert" not found
Sep 10 12:22:17.206719 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:17.206663    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls podName:2395444e-d888-4895-80ad-d263d8676f2f nodeName:}" failed. No retries permitted until 2026-09-10 12:22:17.7066495 +0000 UTC m=+765.487206952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls") pod "maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" (UID: "2395444e-d888-4895-80ad-d263d8676f2f") : secret "maas-api-e2e-inf-a-7576a1-serving-cert" not found
Sep 10 12:22:17.207133 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.207111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.209003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.208981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.244907 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.244886    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82q5f\" (UniqueName: \"kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.710962 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.710933    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.713467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.713442    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") pod \"maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:17.971197 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:17.971125    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:18.260341 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:18.260310    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:22:18.261729 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:22:18.261705    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2395444e_d888_4895_80ad_d263d8676f2f.slice/crio-82089df8db6050f93454def6b7732b0a6859f58636f52a36a267afccfe999028 WatchSource:0}: Error finding container 82089df8db6050f93454def6b7732b0a6859f58636f52a36a267afccfe999028: Status 404 returned error can't find the container with id 82089df8db6050f93454def6b7732b0a6859f58636f52a36a267afccfe999028
Sep 10 12:22:18.368708 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:18.368679    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" event={"ID":"2395444e-d888-4895-80ad-d263d8676f2f","Type":"ContainerStarted","Data":"82089df8db6050f93454def6b7732b0a6859f58636f52a36a267afccfe999028"}
Sep 10 12:22:19.375622 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:19.375586    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" event={"ID":"2395444e-d888-4895-80ad-d263d8676f2f","Type":"ContainerStarted","Data":"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"}
Sep 10 12:22:19.375994 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:19.375665    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:19.408327 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:19.408266    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" podStartSLOduration=1.955908097 podStartE2EDuration="2.408253313s" totalImagesPullingTime="452.345216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:22:19.407711949 +0000 UTC m=+767.188269434" watchObservedRunningTime="2026-09-10 12:22:19.408253313 +0000 UTC m=+767.188810786"
Sep 10 12:22:25.386998 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.386967    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:22:25.488589 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.488562    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:22:25.493095 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.493071    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.496435 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.496409    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-772cfbbb-serving-cert\""
Sep 10 12:22:25.514392 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.514356    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:22:25.573363 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.573337    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.573526 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.573368    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tztt5\" (UniqueName: \"kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.573526 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.573493    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.573621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.573574    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.674219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.674152    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.674219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.674188    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tztt5\" (UniqueName: \"kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.674430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.674233    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.674430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.674280    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.674430 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:25.674285    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-serving-cert: secret "maas-api-e2e-ait-772cfbbb-serving-cert" not found
Sep 10 12:22:25.674430 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:25.674411    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls podName:dee9d3b6-5e81-4717-8d66-058f27238ba3 nodeName:}" failed. No retries permitted until 2026-09-10 12:22:26.174362553 +0000 UTC m=+773.954920009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls") pod "maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3") : secret "maas-api-e2e-ait-772cfbbb-serving-cert" not found
Sep 10 12:22:25.674850 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.674829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.676747 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.676721    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:25.687617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:25.687595    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tztt5\" (UniqueName: \"kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:26.178802 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:26.178771    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:26.178980 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:26.178899    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-serving-cert: secret "maas-api-e2e-ait-772cfbbb-serving-cert" not found
Sep 10 12:22:26.178980 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:26.178966    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls podName:dee9d3b6-5e81-4717-8d66-058f27238ba3 nodeName:}" failed. No retries permitted until 2026-09-10 12:22:27.17895157 +0000 UTC m=+774.959509025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls") pod "maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3") : secret "maas-api-e2e-ait-772cfbbb-serving-cert" not found
Sep 10 12:22:27.189945 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.189909    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:27.192430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.192407    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:27.305241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.305219    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:27.401485 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.401457    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:22:27.413370 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.413342    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:22:27.413581 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.413462    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.417145 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.417128    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-98aa83-serving-cert\""
Sep 10 12:22:27.497573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.497500    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.497573 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.497550    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vnjf\" (UniqueName: \"kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.497718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.497611    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.497718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.497669    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.587091 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.587066    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:22:27.588724 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:22:27.588691    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee9d3b6_5e81_4717_8d66_058f27238ba3.slice/crio-2f90dd9fd8e359b85b9b1019aade83962a0353bbfa3da54d52d07869776bd45b WatchSource:0}: Error finding container 2f90dd9fd8e359b85b9b1019aade83962a0353bbfa3da54d52d07869776bd45b: Status 404 returned error can't find the container with id 2f90dd9fd8e359b85b9b1019aade83962a0353bbfa3da54d52d07869776bd45b
Sep 10 12:22:27.598620 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.598596    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.598897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.598875    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.598974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.598940    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vnjf\" (UniqueName: \"kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.599024 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:27.598977    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-serving-cert: secret "maas-api-e2e-autores-98aa83-serving-cert" not found
Sep 10 12:22:27.599075 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:22:27.599037    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls podName:1a8551be-ec1a-409d-9b0b-23a293d075d5 nodeName:}" failed. No retries permitted until 2026-09-10 12:22:28.099018754 +0000 UTC m=+775.879576205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls") pod "maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" (UID: "1a8551be-ec1a-409d-9b0b-23a293d075d5") : secret "maas-api-e2e-autores-98aa83-serving-cert" not found
Sep 10 12:22:27.599075 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.598984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.599751 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.599727    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.601108 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.601083    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:27.610100 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:27.610082    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vnjf\" (UniqueName: \"kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:28.104294 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.104263    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:28.106474 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.106456    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") pod \"maas-api-e2e-autores-98aa83-557d75d68d-xmtsp\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:28.330045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.330013    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:28.429904 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.429865    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" event={"ID":"dee9d3b6-5e81-4717-8d66-058f27238ba3","Type":"ContainerStarted","Data":"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"}
Sep 10 12:22:28.430054 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.429931    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" event={"ID":"dee9d3b6-5e81-4717-8d66-058f27238ba3","Type":"ContainerStarted","Data":"2f90dd9fd8e359b85b9b1019aade83962a0353bbfa3da54d52d07869776bd45b"}
Sep 10 12:22:28.430131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.430065    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:28.460660 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.460612    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" podStartSLOduration=3.089241614 podStartE2EDuration="3.460598132s" totalImagesPullingTime="371.356518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:22:28.457706398 +0000 UTC m=+776.238263869" watchObservedRunningTime="2026-09-10 12:22:28.460598132 +0000 UTC m=+776.241155605"
Sep 10 12:22:28.611445 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:28.611417    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:22:28.613255 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:22:28.613233    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8551be_ec1a_409d_9b0b_23a293d075d5.slice/crio-8525ffd480ba5483692537f5ffb45187889e06aa63f27267c1757863d0afb3d8 WatchSource:0}: Error finding container 8525ffd480ba5483692537f5ffb45187889e06aa63f27267c1757863d0afb3d8: Status 404 returned error can't find the container with id 8525ffd480ba5483692537f5ffb45187889e06aa63f27267c1757863d0afb3d8
Sep 10 12:22:29.436922 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:29.436886    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" event={"ID":"1a8551be-ec1a-409d-9b0b-23a293d075d5","Type":"ContainerStarted","Data":"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"}
Sep 10 12:22:29.437294 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:29.436947    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" event={"ID":"1a8551be-ec1a-409d-9b0b-23a293d075d5","Type":"ContainerStarted","Data":"8525ffd480ba5483692537f5ffb45187889e06aa63f27267c1757863d0afb3d8"}
Sep 10 12:22:29.437294 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:29.436974    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:29.549292 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:29.549243    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" podStartSLOduration=2.148894157 podStartE2EDuration="2.549225574s" totalImagesPullingTime="400.331417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:22:29.548251807 +0000 UTC m=+777.328809306" watchObservedRunningTime="2026-09-10 12:22:29.549225574 +0000 UTC m=+777.329783049"
Sep 10 12:22:34.442575 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:34.442547    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:22:35.447709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:35.447681    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:22:46.444211 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.444179    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:22:46.447912 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.447893    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.451057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.451030    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fb35b9-serving-cert\""
Sep 10 12:22:46.470508 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.470486    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:22:46.560742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.560712    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65qm5\" (UniqueName: \"kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.560742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.560745    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.560923 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.560869    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.560923 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.560907    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.662051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.661997    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.662235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.662125    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.662235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.662193    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65qm5\" (UniqueName: \"kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.662235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.662212    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.662867 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.662842    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.664631 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.664608    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.664742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.664729    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.677045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.677020    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65qm5\" (UniqueName: \"kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5\") pod \"maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:46.758436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:46.758356    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:47.075534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:47.075460    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:22:47.079103 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:22:47.079077    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc815a526_83d1_4f41_846d_324623f38aa2.slice/crio-7f3906185c92437c1b8598defa9f2f4ca1b0def2b38261b8ce23219255c61907 WatchSource:0}: Error finding container 7f3906185c92437c1b8598defa9f2f4ca1b0def2b38261b8ce23219255c61907: Status 404 returned error can't find the container with id 7f3906185c92437c1b8598defa9f2f4ca1b0def2b38261b8ce23219255c61907
Sep 10 12:22:47.535987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:47.535942    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" event={"ID":"c815a526-83d1-4f41-846d-324623f38aa2","Type":"ContainerStarted","Data":"7f3906185c92437c1b8598defa9f2f4ca1b0def2b38261b8ce23219255c61907"}
Sep 10 12:22:49.547911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:49.547879    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" event={"ID":"c815a526-83d1-4f41-846d-324623f38aa2","Type":"ContainerStarted","Data":"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"}
Sep 10 12:22:49.548257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:49.547945    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:22:49.573907 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:49.573859    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" podStartSLOduration=1.6369468 podStartE2EDuration="3.573842267s" totalImagesPullingTime="1.936895467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:22:49.573526115 +0000 UTC m=+797.354083607" watchObservedRunningTime="2026-09-10 12:22:49.573842267 +0000 UTC m=+797.354399741"
Sep 10 12:22:55.558462 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:22:55.558395    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:23:06.367283 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.367251    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:23:06.367891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.367861    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" podUID="dee9d3b6-5e81-4717-8d66-058f27238ba3" containerName="maas-api" containerID="cri-o://fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65" gracePeriod=30
Sep 10 12:23:06.624157 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.624105    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:23:06.637424 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.637400    2551 generic.go:356] "Generic (PLEG): container finished" podID="dee9d3b6-5e81-4717-8d66-058f27238ba3" containerID="fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65" exitCode=0
Sep 10 12:23:06.637534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.637436    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"
Sep 10 12:23:06.637534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.637447    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" event={"ID":"dee9d3b6-5e81-4717-8d66-058f27238ba3","Type":"ContainerDied","Data":"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"}
Sep 10 12:23:06.637534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.637499    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb" event={"ID":"dee9d3b6-5e81-4717-8d66-058f27238ba3","Type":"ContainerDied","Data":"2f90dd9fd8e359b85b9b1019aade83962a0353bbfa3da54d52d07869776bd45b"}
Sep 10 12:23:06.637534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.637529    2551 scope.go:117] "RemoveContainer" containerID="fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"
Sep 10 12:23:06.647532 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.647478    2551 scope.go:117] "RemoveContainer" containerID="fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"
Sep 10 12:23:06.648047 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:23:06.647997    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65\": container with ID starting with fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65 not found: ID does not exist" containerID="fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"
Sep 10 12:23:06.648047 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.648033    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65"} err="failed to get container status \"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65\": rpc error: code = NotFound desc = could not find container \"fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65\": container with ID starting with fdefd84a4f45864090f39e3cd6f919f84915fe16b09955d72b90f38dcdc1ee65 not found: ID does not exist"
Sep 10 12:23:06.710201 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.710182    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") pod \"dee9d3b6-5e81-4717-8d66-058f27238ba3\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") "
Sep 10 12:23:06.710295 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.710217    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca\") pod \"dee9d3b6-5e81-4717-8d66-058f27238ba3\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") "
Sep 10 12:23:06.710295 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.710264    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs\") pod \"dee9d3b6-5e81-4717-8d66-058f27238ba3\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") "
Sep 10 12:23:06.710295 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.710284    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tztt5\" (UniqueName: \"kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5\") pod \"dee9d3b6-5e81-4717-8d66-058f27238ba3\" (UID: \"dee9d3b6-5e81-4717-8d66-058f27238ba3\") "
Sep 10 12:23:06.710697 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.710661    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dee9d3b6-5e81-4717-8d66-058f27238ba3" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:23:06.712275 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.712252    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dee9d3b6-5e81-4717-8d66-058f27238ba3" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:06.712399 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.712365    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dee9d3b6-5e81-4717-8d66-058f27238ba3" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:06.712603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.712474    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5" (OuterVolumeSpecName: "kube-api-access-tztt5") pod "dee9d3b6-5e81-4717-8d66-058f27238ba3" (UID: "dee9d3b6-5e81-4717-8d66-058f27238ba3"). InnerVolumeSpecName "kube-api-access-tztt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:23:06.811795 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.811769    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:06.811795 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.811793    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee9d3b6-5e81-4717-8d66-058f27238ba3-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:06.811795 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.811802    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dee9d3b6-5e81-4717-8d66-058f27238ba3-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:06.811986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.811811    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tztt5\" (UniqueName: \"kubernetes.io/projected/dee9d3b6-5e81-4717-8d66-058f27238ba3-kube-api-access-tztt5\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:06.955405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.955318    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:23:06.958591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:06.958569    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-772cfbbb-6f95f759b4-fw5gb"]
Sep 10 12:23:08.723099 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:08.723067    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dee9d3b6-5e81-4717-8d66-058f27238ba3" path="/var/lib/kubelet/pods/dee9d3b6-5e81-4717-8d66-058f27238ba3/volumes"
Sep 10 12:23:14.337182 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.337153    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:14.337718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.337623    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee9d3b6-5e81-4717-8d66-058f27238ba3" containerName="maas-api"
Sep 10 12:23:14.337718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.337634    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee9d3b6-5e81-4717-8d66-058f27238ba3" containerName="maas-api"
Sep 10 12:23:14.337718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.337697    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee9d3b6-5e81-4717-8d66-058f27238ba3" containerName="maas-api"
Sep 10 12:23:14.342485 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.342468    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.345230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.345211    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-41f030dc-serving-cert\""
Sep 10 12:23:14.359948 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.359921    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:14.375593 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.375569    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.375713 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.375603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.375713 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.375647    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5h2t\" (UniqueName: \"kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.375825 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.375776    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.476643 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.476619    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5h2t\" (UniqueName: \"kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.476764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.476671    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.476764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.476693    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.476764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.476713    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.476922 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:23:14.476831    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-serving-cert: secret "maas-api-e2e-migrate-41f030dc-serving-cert" not found
Sep 10 12:23:14.476922 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:23:14.476892    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls podName:cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e nodeName:}" failed. No retries permitted until 2026-09-10 12:23:14.976874769 +0000 UTC m=+822.757432225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls") pod "maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" (UID: "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e") : secret "maas-api-e2e-migrate-41f030dc-serving-cert" not found
Sep 10 12:23:14.477338 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.477316    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.479095 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.479074    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.491312 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.491288    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5h2t\" (UniqueName: \"kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.981649 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.981617    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:14.983889 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:14.983870    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") pod \"maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:15.255462 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:15.255395    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:15.528883 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:15.528859    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:15.530704 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:23:15.530681    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcf9a0c_4b20_45f7_b6e5_36c7b1610a0e.slice/crio-a203336cc64319caf2129464a0e0fadb66d875d939ceb3f26a50135d2e6e987b WatchSource:0}: Error finding container a203336cc64319caf2129464a0e0fadb66d875d939ceb3f26a50135d2e6e987b: Status 404 returned error can't find the container with id a203336cc64319caf2129464a0e0fadb66d875d939ceb3f26a50135d2e6e987b
Sep 10 12:23:15.684759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:15.684726    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" event={"ID":"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e","Type":"ContainerStarted","Data":"a203336cc64319caf2129464a0e0fadb66d875d939ceb3f26a50135d2e6e987b"}
Sep 10 12:23:16.696615 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:16.696576    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" event={"ID":"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e","Type":"ContainerStarted","Data":"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"}
Sep 10 12:23:16.697065 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:16.696649    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:16.716675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:16.716630    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" podStartSLOduration=2.399043563 podStartE2EDuration="2.716615548s" totalImagesPullingTime="317.571985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:23:16.716326244 +0000 UTC m=+824.496883739" watchObservedRunningTime="2026-09-10 12:23:16.716615548 +0000 UTC m=+824.497173062"
Sep 10 12:23:22.708115 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:22.708078    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:55.170932 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.170892    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:55.171855 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.171789    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" podUID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" containerName="maas-api" containerID="cri-o://4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e" gracePeriod=30
Sep 10 12:23:55.422178 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.422125    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:55.606881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.606840    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") pod \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") "
Sep 10 12:23:55.607059 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.606954    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca\") pod \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") "
Sep 10 12:23:55.607059 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.607010    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5h2t\" (UniqueName: \"kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t\") pod \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") "
Sep 10 12:23:55.607174 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.607065    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs\") pod \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\" (UID: \"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e\") "
Sep 10 12:23:55.607600 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.607564    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" (UID: "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:23:55.609878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.609845    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" (UID: "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:55.609998 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.609915    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" (UID: "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:55.610049 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.610017    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t" (OuterVolumeSpecName: "kube-api-access-p5h2t") pod "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" (UID: "cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e"). InnerVolumeSpecName "kube-api-access-p5h2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:23:55.708069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.708010    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:55.708069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.708034    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:55.708069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.708043    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5h2t\" (UniqueName: \"kubernetes.io/projected/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-kube-api-access-p5h2t\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:55.708069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.708051    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:23:55.900747 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.900717    2551 generic.go:356] "Generic (PLEG): container finished" podID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" containerID="4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e" exitCode=0
Sep 10 12:23:55.900928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.900769    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"
Sep 10 12:23:55.900928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.900789    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" event={"ID":"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e","Type":"ContainerDied","Data":"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"}
Sep 10 12:23:55.900928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.900829    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp" event={"ID":"cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e","Type":"ContainerDied","Data":"a203336cc64319caf2129464a0e0fadb66d875d939ceb3f26a50135d2e6e987b"}
Sep 10 12:23:55.900928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.900850    2551 scope.go:117] "RemoveContainer" containerID="4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"
Sep 10 12:23:55.910500 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.910449    2551 scope.go:117] "RemoveContainer" containerID="4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"
Sep 10 12:23:55.910715 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:23:55.910691    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e\": container with ID starting with 4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e not found: ID does not exist" containerID="4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"
Sep 10 12:23:55.910793 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.910726    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e"} err="failed to get container status \"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e\": rpc error: code = NotFound desc = could not find container \"4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e\": container with ID starting with 4bb4b3431f532bc1c14616259ea2839b0c77425486f7128c3098611500dc979e not found: ID does not exist"
Sep 10 12:23:55.924400 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.924357    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:55.935456 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:55.929397    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41f030dc-56d858b489-wd7zp"]
Sep 10 12:23:56.725275 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:23:56.725244    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" path="/var/lib/kubelet/pods/cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e/volumes"
Sep 10 12:24:00.164601 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.164567    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:00.164983 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.164971    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" containerName="maas-api"
Sep 10 12:24:00.164983 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.164984    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" containerName="maas-api"
Sep 10 12:24:00.165067 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.165053    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcf9a0c-4b20-45f7-b6e5-36c7b1610a0e" containerName="maas-api"
Sep 10 12:24:00.168136 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.168116    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.170884 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.170867    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-67fd52b1-serving-cert\""
Sep 10 12:24:00.182885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.182828    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:00.343042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.343010    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.343216 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.343047    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.343216 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.343167    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjgpj\" (UniqueName: \"kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.343216 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.343214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.443821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.443745    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.443821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.443774    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.444001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.443832    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjgpj\" (UniqueName: \"kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.444001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.443855    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.444555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.444527    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.446220 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.446199    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.446321 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.446208    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.452769 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.452745    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjgpj\" (UniqueName: \"kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj\") pod \"maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.478229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.478206    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:00.759536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.759509    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:00.761491 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:24:00.761466    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddacf1c_5c49_4dcd_867f_0639f37e2126.slice/crio-84c4213cbe929deec9b93355908f7a7719c03b2a1f14b08024ff45be9e06d104 WatchSource:0}: Error finding container 84c4213cbe929deec9b93355908f7a7719c03b2a1f14b08024ff45be9e06d104: Status 404 returned error can't find the container with id 84c4213cbe929deec9b93355908f7a7719c03b2a1f14b08024ff45be9e06d104
Sep 10 12:24:00.930408 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:00.930366    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" event={"ID":"addacf1c-5c49-4dcd-867f-0639f37e2126","Type":"ContainerStarted","Data":"84c4213cbe929deec9b93355908f7a7719c03b2a1f14b08024ff45be9e06d104"}
Sep 10 12:24:01.938242 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:01.938204    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" event={"ID":"addacf1c-5c49-4dcd-867f-0639f37e2126","Type":"ContainerStarted","Data":"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"}
Sep 10 12:24:01.938668 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:01.938272    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:01.964954 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:01.964892    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" podStartSLOduration=1.43359711 podStartE2EDuration="1.964874875s" totalImagesPullingTime="531.277765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:01.957594136 +0000 UTC m=+869.738151717" watchObservedRunningTime="2026-09-10 12:24:01.964874875 +0000 UTC m=+869.745432347"
Sep 10 12:24:07.950506 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:07.950478    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:11.909096 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:11.909065    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:24:11.909525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:11.909239    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" podUID="1a8551be-ec1a-409d-9b0b-23a293d075d5" containerName="maas-api" containerID="cri-o://41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940" gracePeriod=30
Sep 10 12:24:12.192142 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.192098    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:24:12.242820 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.242794    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vnjf\" (UniqueName: \"kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf\") pod \"1a8551be-ec1a-409d-9b0b-23a293d075d5\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") "
Sep 10 12:24:12.242952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.242828    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs\") pod \"1a8551be-ec1a-409d-9b0b-23a293d075d5\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") "
Sep 10 12:24:12.242952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.242933    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca\") pod \"1a8551be-ec1a-409d-9b0b-23a293d075d5\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") "
Sep 10 12:24:12.243073 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.242985    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") pod \"1a8551be-ec1a-409d-9b0b-23a293d075d5\" (UID: \"1a8551be-ec1a-409d-9b0b-23a293d075d5\") "
Sep 10 12:24:12.243421 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.243392    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a8551be-ec1a-409d-9b0b-23a293d075d5" (UID: "1a8551be-ec1a-409d-9b0b-23a293d075d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:24:12.244952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.244932    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1a8551be-ec1a-409d-9b0b-23a293d075d5" (UID: "1a8551be-ec1a-409d-9b0b-23a293d075d5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:12.245238 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.245204    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a8551be-ec1a-409d-9b0b-23a293d075d5" (UID: "1a8551be-ec1a-409d-9b0b-23a293d075d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:12.245452 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.245434    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf" (OuterVolumeSpecName: "kube-api-access-6vnjf") pod "1a8551be-ec1a-409d-9b0b-23a293d075d5" (UID: "1a8551be-ec1a-409d-9b0b-23a293d075d5"). InnerVolumeSpecName "kube-api-access-6vnjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:24:12.344614 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.344590    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:12.344614 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.344611    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8551be-ec1a-409d-9b0b-23a293d075d5-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:12.344749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.344622    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8551be-ec1a-409d-9b0b-23a293d075d5-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:12.344749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:12.344630    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vnjf\" (UniqueName: \"kubernetes.io/projected/1a8551be-ec1a-409d-9b0b-23a293d075d5-kube-api-access-6vnjf\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:13.004467 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.004401    2551 generic.go:356] "Generic (PLEG): container finished" podID="1a8551be-ec1a-409d-9b0b-23a293d075d5" containerID="41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940" exitCode=0
Sep 10 12:24:13.004821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.004472    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" event={"ID":"1a8551be-ec1a-409d-9b0b-23a293d075d5","Type":"ContainerDied","Data":"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"}
Sep 10 12:24:13.004821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.004485    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"
Sep 10 12:24:13.004821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.004507    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp" event={"ID":"1a8551be-ec1a-409d-9b0b-23a293d075d5","Type":"ContainerDied","Data":"8525ffd480ba5483692537f5ffb45187889e06aa63f27267c1757863d0afb3d8"}
Sep 10 12:24:13.004821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.004529    2551 scope.go:117] "RemoveContainer" containerID="41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"
Sep 10 12:24:13.013029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.013013    2551 scope.go:117] "RemoveContainer" containerID="41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"
Sep 10 12:24:13.013256 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:24:13.013240    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940\": container with ID starting with 41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940 not found: ID does not exist" containerID="41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"
Sep 10 12:24:13.013296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.013263    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940"} err="failed to get container status \"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940\": rpc error: code = NotFound desc = could not find container \"41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940\": container with ID starting with 41c8bf6b25be27db93db2b07cfa7c13c63c59a217f56733b64fd933a50ca2940 not found: ID does not exist"
Sep 10 12:24:13.031087 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.031069    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:24:13.034496 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:13.034477    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-98aa83-557d75d68d-xmtsp"]
Sep 10 12:24:14.724212 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:14.724181    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8551be-ec1a-409d-9b0b-23a293d075d5" path="/var/lib/kubelet/pods/1a8551be-ec1a-409d-9b0b-23a293d075d5/volumes"
Sep 10 12:24:25.727033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.725240    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:24:25.727033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.725916    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8551be-ec1a-409d-9b0b-23a293d075d5" containerName="maas-api"
Sep 10 12:24:25.727033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.725934    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8551be-ec1a-409d-9b0b-23a293d075d5" containerName="maas-api"
Sep 10 12:24:25.727033 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.726028    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8551be-ec1a-409d-9b0b-23a293d075d5" containerName="maas-api"
Sep 10 12:24:25.731788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.731766    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.734310 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.734287    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-f615cc-serving-cert\""
Sep 10 12:24:25.739550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.739475    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:24:25.854873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.854842    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.854996 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.854965    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.855053 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.855002    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddgwg\" (UniqueName: \"kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.855111 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.855053    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.956221 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.956192    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.956221 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.956222    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddgwg\" (UniqueName: \"kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.956469 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.956261    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.956469 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.956310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.956469 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:24:25.956351    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-serving-cert: secret "maas-api-e2e-explicit-f615cc-serving-cert" not found
Sep 10 12:24:25.956469 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:24:25.956456    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls podName:e8213bae-a40c-49d0-94ab-9829b2649a2b nodeName:}" failed. No retries permitted until 2026-09-10 12:24:26.4564362 +0000 UTC m=+894.236993651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls") pod "maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" (UID: "e8213bae-a40c-49d0-94ab-9829b2649a2b") : secret "maas-api-e2e-explicit-f615cc-serving-cert" not found
Sep 10 12:24:25.957005 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.956984    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.958789 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.958766    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:25.971209 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:25.971175    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddgwg\" (UniqueName: \"kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:26.461510 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:26.461479    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:26.463835 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:26.463811    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") pod \"maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:26.664708 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:26.664675    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:26.943151 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:26.943121    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:24:26.944476 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:24:26.944452    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8213bae_a40c_49d0_94ab_9829b2649a2b.slice/crio-3de026fd4232528c5a66a3d1abfb60b89f4b7fed8e24814e3291c63a104ac3e7 WatchSource:0}: Error finding container 3de026fd4232528c5a66a3d1abfb60b89f4b7fed8e24814e3291c63a104ac3e7: Status 404 returned error can't find the container with id 3de026fd4232528c5a66a3d1abfb60b89f4b7fed8e24814e3291c63a104ac3e7
Sep 10 12:24:27.080943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:27.080915    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" event={"ID":"e8213bae-a40c-49d0-94ab-9829b2649a2b","Type":"ContainerStarted","Data":"3de026fd4232528c5a66a3d1abfb60b89f4b7fed8e24814e3291c63a104ac3e7"}
Sep 10 12:24:28.088550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:28.088512    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" event={"ID":"e8213bae-a40c-49d0-94ab-9829b2649a2b","Type":"ContainerStarted","Data":"5b6488b7b53112e2813eca8a602deb3fdcec65456f455c6338530fb7e40ef051"}
Sep 10 12:24:28.088897 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:28.088692    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:28.109045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:28.108999    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" podStartSLOduration=2.639375672 podStartE2EDuration="3.108985747s" totalImagesPullingTime="469.610075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:28.108974931 +0000 UTC m=+895.889532424" watchObservedRunningTime="2026-09-10 12:24:28.108985747 +0000 UTC m=+895.889543198"
Sep 10 12:24:31.202846 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.202812    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:31.203601 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.203571    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" podUID="addacf1c-5c49-4dcd-867f-0639f37e2126" containerName="maas-api" containerID="cri-o://66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c" gracePeriod=30
Sep 10 12:24:31.466037 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.465983    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:31.608210 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.608182    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjgpj\" (UniqueName: \"kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj\") pod \"addacf1c-5c49-4dcd-867f-0639f37e2126\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") "
Sep 10 12:24:31.608360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.608220    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls\") pod \"addacf1c-5c49-4dcd-867f-0639f37e2126\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") "
Sep 10 12:24:31.608360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.608281    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs\") pod \"addacf1c-5c49-4dcd-867f-0639f37e2126\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") "
Sep 10 12:24:31.608360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.608327    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca\") pod \"addacf1c-5c49-4dcd-867f-0639f37e2126\" (UID: \"addacf1c-5c49-4dcd-867f-0639f37e2126\") "
Sep 10 12:24:31.608745 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.608717    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "addacf1c-5c49-4dcd-867f-0639f37e2126" (UID: "addacf1c-5c49-4dcd-867f-0639f37e2126"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:24:31.610241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.610215    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "addacf1c-5c49-4dcd-867f-0639f37e2126" (UID: "addacf1c-5c49-4dcd-867f-0639f37e2126"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:31.610241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.610216    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj" (OuterVolumeSpecName: "kube-api-access-zjgpj") pod "addacf1c-5c49-4dcd-867f-0639f37e2126" (UID: "addacf1c-5c49-4dcd-867f-0639f37e2126"). InnerVolumeSpecName "kube-api-access-zjgpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:24:31.610393 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.610302    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "addacf1c-5c49-4dcd-867f-0639f37e2126" (UID: "addacf1c-5c49-4dcd-867f-0639f37e2126"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:31.709833 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.709806    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/addacf1c-5c49-4dcd-867f-0639f37e2126-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:31.709833 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.709832    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjgpj\" (UniqueName: \"kubernetes.io/projected/addacf1c-5c49-4dcd-867f-0639f37e2126-kube-api-access-zjgpj\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:31.709950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.709841    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:31.709950 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:31.709849    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addacf1c-5c49-4dcd-867f-0639f37e2126-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:24:32.109449 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.109415    2551 generic.go:356] "Generic (PLEG): container finished" podID="addacf1c-5c49-4dcd-867f-0639f37e2126" containerID="66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c" exitCode=0
Sep 10 12:24:32.109598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.109455    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"
Sep 10 12:24:32.109598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.109465    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" event={"ID":"addacf1c-5c49-4dcd-867f-0639f37e2126","Type":"ContainerDied","Data":"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"}
Sep 10 12:24:32.109598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.109503    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h" event={"ID":"addacf1c-5c49-4dcd-867f-0639f37e2126","Type":"ContainerDied","Data":"84c4213cbe929deec9b93355908f7a7719c03b2a1f14b08024ff45be9e06d104"}
Sep 10 12:24:32.109598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.109523    2551 scope.go:117] "RemoveContainer" containerID="66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"
Sep 10 12:24:32.118675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.118655    2551 scope.go:117] "RemoveContainer" containerID="66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"
Sep 10 12:24:32.118910 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:24:32.118894    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c\": container with ID starting with 66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c not found: ID does not exist" containerID="66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"
Sep 10 12:24:32.118964 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.118917    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c"} err="failed to get container status \"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c\": rpc error: code = NotFound desc = could not find container \"66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c\": container with ID starting with 66fc0e198ee080e7ca7ec674808d6e67f091cb3c3f17350ec54a880197d98f0c not found: ID does not exist"
Sep 10 12:24:32.131660 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.131641    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:32.138928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.138909    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67fd52b1-6877c8f566-9nw9h"]
Sep 10 12:24:32.671297 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.671267    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:24:32.672289 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.672267    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:24:32.676352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.676333    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:24:32.677997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.677977    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:24:32.727788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:32.727762    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="addacf1c-5c49-4dcd-867f-0639f37e2126" path="/var/lib/kubelet/pods/addacf1c-5c49-4dcd-867f-0639f37e2126/volumes"
Sep 10 12:24:34.098548 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:34.098518    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:24:37.546199 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.546169    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:24:37.546626 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.546613    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="addacf1c-5c49-4dcd-867f-0639f37e2126" containerName="maas-api"
Sep 10 12:24:37.546670 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.546630    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="addacf1c-5c49-4dcd-867f-0639f37e2126" containerName="maas-api"
Sep 10 12:24:37.546725 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.546716    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="addacf1c-5c49-4dcd-867f-0639f37e2126" containerName="maas-api"
Sep 10 12:24:37.551668 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.551646    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.554937 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.554910    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9e0ca8ed-serving-cert\""
Sep 10 12:24:37.565864 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.565843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:24:37.659488 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.659461    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.659619 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.659505    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9qjf\" (UniqueName: \"kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.659619 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.659554    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.659708 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.659631    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.761029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.761000    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.761160 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.761070    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.761160 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.761128    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.761242 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.761173    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9qjf\" (UniqueName: \"kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.761785 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.761761    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.763445 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.763422    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.763536 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.763458    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.778645 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.778623    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9qjf\" (UniqueName: \"kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf\") pod \"maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:37.862644 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:37.862621    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:38.159981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:38.159959    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:24:38.161943 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:24:38.161893    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16a4c2b_4b94_41e0_b175_03e505cfb47f.slice/crio-a2e628539a52e8520279188e673ec040c0ef73c307edf1b83ee4d0b37aaca88f WatchSource:0}: Error finding container a2e628539a52e8520279188e673ec040c0ef73c307edf1b83ee4d0b37aaca88f: Status 404 returned error can't find the container with id a2e628539a52e8520279188e673ec040c0ef73c307edf1b83ee4d0b37aaca88f
Sep 10 12:24:39.152457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:39.152413    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" event={"ID":"e16a4c2b-4b94-41e0-b175-03e505cfb47f","Type":"ContainerStarted","Data":"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"}
Sep 10 12:24:39.152927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:39.152475    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" event={"ID":"e16a4c2b-4b94-41e0-b175-03e505cfb47f","Type":"ContainerStarted","Data":"a2e628539a52e8520279188e673ec040c0ef73c307edf1b83ee4d0b37aaca88f"}
Sep 10 12:24:39.152927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:39.152557    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:24:39.180196 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:39.180153    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" podStartSLOduration=1.76150318 podStartE2EDuration="2.180140303s" totalImagesPullingTime="418.637123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:39.178033297 +0000 UTC m=+906.958590787" watchObservedRunningTime="2026-09-10 12:24:39.180140303 +0000 UTC m=+906.960697783"
Sep 10 12:24:45.164079 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:24:45.164054    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:25:08.472584 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.472546    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:25:08.473027 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.472733    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" podUID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" containerName="maas-api" containerID="cri-o://117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55" gracePeriod=30
Sep 10 12:25:08.737291 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.737236    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:25:08.911184 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911151    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9qjf\" (UniqueName: \"kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf\") pod \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") "
Sep 10 12:25:08.911357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911209    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca\") pod \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") "
Sep 10 12:25:08.911357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911239    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls\") pod \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") "
Sep 10 12:25:08.911357 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911316    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs\") pod \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\" (UID: \"e16a4c2b-4b94-41e0-b175-03e505cfb47f\") "
Sep 10 12:25:08.911655 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911622    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e16a4c2b-4b94-41e0-b175-03e505cfb47f" (UID: "e16a4c2b-4b94-41e0-b175-03e505cfb47f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:25:08.911779 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.911752    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16a4c2b-4b94-41e0-b175-03e505cfb47f-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:25:08.913363 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.913335    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e16a4c2b-4b94-41e0-b175-03e505cfb47f" (UID: "e16a4c2b-4b94-41e0-b175-03e505cfb47f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:25:08.913499 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.913360    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e16a4c2b-4b94-41e0-b175-03e505cfb47f" (UID: "e16a4c2b-4b94-41e0-b175-03e505cfb47f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:25:08.913499 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:08.913414    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf" (OuterVolumeSpecName: "kube-api-access-w9qjf") pod "e16a4c2b-4b94-41e0-b175-03e505cfb47f" (UID: "e16a4c2b-4b94-41e0-b175-03e505cfb47f"). InnerVolumeSpecName "kube-api-access-w9qjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:25:09.012750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.012690    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9qjf\" (UniqueName: \"kubernetes.io/projected/e16a4c2b-4b94-41e0-b175-03e505cfb47f-kube-api-access-w9qjf\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:25:09.012750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.012714    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:25:09.012750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.012724    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16a4c2b-4b94-41e0-b175-03e505cfb47f-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:25:09.317914 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.317849    2551 generic.go:356] "Generic (PLEG): container finished" podID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" containerID="117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55" exitCode=0
Sep 10 12:25:09.317914 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.317890    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"
Sep 10 12:25:09.318090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.317936    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" event={"ID":"e16a4c2b-4b94-41e0-b175-03e505cfb47f","Type":"ContainerDied","Data":"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"}
Sep 10 12:25:09.318090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.317984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd" event={"ID":"e16a4c2b-4b94-41e0-b175-03e505cfb47f","Type":"ContainerDied","Data":"a2e628539a52e8520279188e673ec040c0ef73c307edf1b83ee4d0b37aaca88f"}
Sep 10 12:25:09.318090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.318012    2551 scope.go:117] "RemoveContainer" containerID="117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"
Sep 10 12:25:09.328940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.328914    2551 scope.go:117] "RemoveContainer" containerID="117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"
Sep 10 12:25:09.329315 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:25:09.329298    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55\": container with ID starting with 117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55 not found: ID does not exist" containerID="117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"
Sep 10 12:25:09.329415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.329322    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55"} err="failed to get container status \"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55\": rpc error: code = NotFound desc = could not find container \"117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55\": container with ID starting with 117b37d68fd482f3d0533c2c86acf468b6a6fe5de7f4b0f0679ed129a387bc55 not found: ID does not exist"
Sep 10 12:25:09.372068 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.372047    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:25:09.393317 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:09.393294    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9e0ca8ed-77c87cd775-gfwqd"]
Sep 10 12:25:10.723685 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:10.723652    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" path="/var/lib/kubelet/pods/e16a4c2b-4b94-41e0-b175-03e505cfb47f/volumes"
Sep 10 12:25:30.559747 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.559714    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:25:30.560156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.560135    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" containerName="maas-api"
Sep 10 12:25:30.560156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.560156    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" containerName="maas-api"
Sep 10 12:25:30.560228 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.560217    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16a4c2b-4b94-41e0-b175-03e505cfb47f" containerName="maas-api"
Sep 10 12:25:30.563195 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.563177    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.574545 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.574527    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-f615cc\"/\"kube-root-ca.crt\""
Sep 10 12:25:30.574688 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.574584    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-f615cc\"/\"openshift-service-ca.crt\""
Sep 10 12:25:30.575232 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.575217    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-f615cc\"/\"default-dockercfg-qz9lx\""
Sep 10 12:25:30.575793 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.575777    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-f615cc\"/\"explicit-model-f615cc-kserve-self-signed-certs\""
Sep 10 12:25:30.599291 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.599268    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:25:30.689855 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.689833    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.689966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.689862    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-894wl\" (UniqueName: \"kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.689966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.689883    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.690055 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.689995    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.690055 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.690024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.690152 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.690061    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.790939 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.790918    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.790948    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.790973    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791004    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791025    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-894wl\" (UniqueName: \"kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791045    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791335 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791317    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791428 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791363    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.791492 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.791457    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.793226 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.793203    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.793530 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.793514    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.805562 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.805537    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-894wl\" (UniqueName: \"kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl\") pod \"explicit-model-f615cc-kserve-6cd478d98f-77scp\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") " pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:30.872142 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:30.872119    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:31.171718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:31.171688    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:25:31.173425 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:25:31.173395    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod474eee1d_98e1_4247_bbb5_4bc1bf5f878d.slice/crio-6f85a02566823445fc8d57f9e52ee04c07e2e309cb5b308d7602e761ca44e8c3 WatchSource:0}: Error finding container 6f85a02566823445fc8d57f9e52ee04c07e2e309cb5b308d7602e761ca44e8c3: Status 404 returned error can't find the container with id 6f85a02566823445fc8d57f9e52ee04c07e2e309cb5b308d7602e761ca44e8c3
Sep 10 12:25:31.435665 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:31.435587    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" event={"ID":"474eee1d-98e1-4247-bbb5-4bc1bf5f878d","Type":"ContainerStarted","Data":"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"}
Sep 10 12:25:31.435665 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:31.435625    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" event={"ID":"474eee1d-98e1-4247-bbb5-4bc1bf5f878d","Type":"ContainerStarted","Data":"6f85a02566823445fc8d57f9e52ee04c07e2e309cb5b308d7602e761ca44e8c3"}
Sep 10 12:25:31.475666 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:31.475621    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" podStartSLOduration=1.475606266 podStartE2EDuration="1.475606266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:25:31.475599196 +0000 UTC m=+959.256156680" watchObservedRunningTime="2026-09-10 12:25:31.475606266 +0000 UTC m=+959.256163738"
Sep 10 12:25:40.872942 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:40.872912    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:40.873405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:40.872955    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:40.886132 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:40.886110    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:41.501285 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:41.501255    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:25:50.671742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:25:50.671711    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:26:01.405821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.405733    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:26:01.406258 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.405973    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" podUID="e8213bae-a40c-49d0-94ab-9829b2649a2b" containerName="maas-api" containerID="cri-o://5b6488b7b53112e2813eca8a602deb3fdcec65456f455c6338530fb7e40ef051" gracePeriod=30
Sep 10 12:26:01.595836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.595801    2551 generic.go:356] "Generic (PLEG): container finished" podID="e8213bae-a40c-49d0-94ab-9829b2649a2b" containerID="5b6488b7b53112e2813eca8a602deb3fdcec65456f455c6338530fb7e40ef051" exitCode=0
Sep 10 12:26:01.596039 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.595874    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" event={"ID":"e8213bae-a40c-49d0-94ab-9829b2649a2b","Type":"ContainerDied","Data":"5b6488b7b53112e2813eca8a602deb3fdcec65456f455c6338530fb7e40ef051"}
Sep 10 12:26:01.650527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.650505    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:26:01.843169 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.843142    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca\") pod \"e8213bae-a40c-49d0-94ab-9829b2649a2b\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") "
Sep 10 12:26:01.843332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.843178    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") pod \"e8213bae-a40c-49d0-94ab-9829b2649a2b\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") "
Sep 10 12:26:01.843332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.843235    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddgwg\" (UniqueName: \"kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg\") pod \"e8213bae-a40c-49d0-94ab-9829b2649a2b\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") "
Sep 10 12:26:01.843332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.843260    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs\") pod \"e8213bae-a40c-49d0-94ab-9829b2649a2b\" (UID: \"e8213bae-a40c-49d0-94ab-9829b2649a2b\") "
Sep 10 12:26:01.843580 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.843552    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8213bae-a40c-49d0-94ab-9829b2649a2b" (UID: "e8213bae-a40c-49d0-94ab-9829b2649a2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:26:01.845325 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.845297    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8213bae-a40c-49d0-94ab-9829b2649a2b" (UID: "e8213bae-a40c-49d0-94ab-9829b2649a2b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:01.845482 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.845327    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8213bae-a40c-49d0-94ab-9829b2649a2b" (UID: "e8213bae-a40c-49d0-94ab-9829b2649a2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:01.845482 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.845400    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg" (OuterVolumeSpecName: "kube-api-access-ddgwg") pod "e8213bae-a40c-49d0-94ab-9829b2649a2b" (UID: "e8213bae-a40c-49d0-94ab-9829b2649a2b"). InnerVolumeSpecName "kube-api-access-ddgwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:26:01.944767 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.944741    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8213bae-a40c-49d0-94ab-9829b2649a2b-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:01.944767 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.944765    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:01.944905 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.944775    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddgwg\" (UniqueName: \"kubernetes.io/projected/e8213bae-a40c-49d0-94ab-9829b2649a2b-kube-api-access-ddgwg\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:01.944905 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:01.944785    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8213bae-a40c-49d0-94ab-9829b2649a2b-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:02.602395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.602345    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"
Sep 10 12:26:02.602395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.602363    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd" event={"ID":"e8213bae-a40c-49d0-94ab-9829b2649a2b","Type":"ContainerDied","Data":"3de026fd4232528c5a66a3d1abfb60b89f4b7fed8e24814e3291c63a104ac3e7"}
Sep 10 12:26:02.602858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.602445    2551 scope.go:117] "RemoveContainer" containerID="5b6488b7b53112e2813eca8a602deb3fdcec65456f455c6338530fb7e40ef051"
Sep 10 12:26:02.632310 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.632283    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:26:02.635760 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.635736    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f615cc-9dc9b4f-cvfxd"]
Sep 10 12:26:02.723429 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:02.723392    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8213bae-a40c-49d0-94ab-9829b2649a2b" path="/var/lib/kubelet/pods/e8213bae-a40c-49d0-94ab-9829b2649a2b/volumes"
Sep 10 12:26:05.690648 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.690604    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" podUID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" containerName="main" containerID="cri-o://390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757" gracePeriod=45
Sep 10 12:26:05.961758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.961734    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:26:05.981228 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981203    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.981426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981245    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.981426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981266    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.981426 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981417    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.981676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981464    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-894wl\" (UniqueName: \"kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.981676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981505    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home" (OuterVolumeSpecName: "home") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:05.981867 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981842    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache" (OuterVolumeSpecName: "model-cache") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:05.982014 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.981992    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir\") pod \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\" (UID: \"474eee1d-98e1-4247-bbb5-4bc1bf5f878d\") "
Sep 10 12:26:05.982327 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.982305    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:05.982440 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.982329    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:05.982598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.982576    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:05.985391 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.985342    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:05.985886 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.985853    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm" (OuterVolumeSpecName: "dshm") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:05.985973 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:05.985926    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl" (OuterVolumeSpecName: "kube-api-access-894wl") pod "474eee1d-98e1-4247-bbb5-4bc1bf5f878d" (UID: "474eee1d-98e1-4247-bbb5-4bc1bf5f878d"). InnerVolumeSpecName "kube-api-access-894wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:26:06.083230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.083169    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:06.083230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.083224    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:06.083230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.083242    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-894wl\" (UniqueName: \"kubernetes.io/projected/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-kube-api-access-894wl\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:06.083552 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.083253    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474eee1d-98e1-4247-bbb5-4bc1bf5f878d-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:06.632588 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.632554    2551 generic.go:356] "Generic (PLEG): container finished" podID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" containerID="390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757" exitCode=0
Sep 10 12:26:06.632790 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.632597    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" event={"ID":"474eee1d-98e1-4247-bbb5-4bc1bf5f878d","Type":"ContainerDied","Data":"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"}
Sep 10 12:26:06.632790 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.632600    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"
Sep 10 12:26:06.632790 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.632624    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp" event={"ID":"474eee1d-98e1-4247-bbb5-4bc1bf5f878d","Type":"ContainerDied","Data":"6f85a02566823445fc8d57f9e52ee04c07e2e309cb5b308d7602e761ca44e8c3"}
Sep 10 12:26:06.632790 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.632641    2551 scope.go:117] "RemoveContainer" containerID="390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"
Sep 10 12:26:06.642121 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.642103    2551 scope.go:117] "RemoveContainer" containerID="390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"
Sep 10 12:26:06.642407 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:06.642369    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757\": container with ID starting with 390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757 not found: ID does not exist" containerID="390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"
Sep 10 12:26:06.642507 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.642414    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757"} err="failed to get container status \"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757\": rpc error: code = NotFound desc = could not find container \"390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757\": container with ID starting with 390b199194be47a7f0ec89cb5f1f554ac0f426a3695903792a0a3a9a88985757 not found: ID does not exist"
Sep 10 12:26:06.660305 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.660280    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:26:06.662761 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.662742    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-f615cc/explicit-model-f615cc-kserve-6cd478d98f-77scp"]
Sep 10 12:26:06.729822 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:06.729789    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" path="/var/lib/kubelet/pods/474eee1d-98e1-4247-bbb5-4bc1bf5f878d/volumes"
Sep 10 12:26:22.067223 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067177    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067605    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" containerName="main"
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067618    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" containerName="main"
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067628    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8213bae-a40c-49d0-94ab-9829b2649a2b" containerName="maas-api"
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067634    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8213bae-a40c-49d0-94ab-9829b2649a2b" containerName="maas-api"
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067690    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="474eee1d-98e1-4247-bbb5-4bc1bf5f878d" containerName="main"
Sep 10 12:26:22.067752 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.067697    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8213bae-a40c-49d0-94ab-9829b2649a2b" containerName="maas-api"
Sep 10 12:26:22.072192 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.072173    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.075861 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.075839    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1d8e56\"/\"kube-root-ca.crt\""
Sep 10 12:26:22.075981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.075861    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1d8e56\"/\"default-dockercfg-9hjrr\""
Sep 10 12:26:22.075981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.075887    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1d8e56\"/\"openshift-service-ca.crt\""
Sep 10 12:26:22.075981 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.075933    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1d8e56\"/\"noten-model-1d8e56-kserve-self-signed-certs\""
Sep 10 12:26:22.082916 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.082895    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:22.124906 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.124881    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdgjp\" (UniqueName: \"kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.125022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.124910    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.125022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.124932    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.125094 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.125041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.125094 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.125063    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.125160 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.125121    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226215 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226194    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226245    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226305    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdgjp\" (UniqueName: \"kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226330    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226593 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226444    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226593 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226527    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226697 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226606    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226697 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226621    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.226826 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.226801    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.228485 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.228467    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.228829 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.228811    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.239331 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.239307    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdgjp\" (UniqueName: \"kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp\") pod \"noten-model-1d8e56-kserve-7ddddb98bc-bzl6x\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") " pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.381741 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.381721    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:22.653711 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.653687    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:22.655523 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:26:22.655499    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ba4db76_c3f1_482e_954f_d88cbafe472a.slice/crio-034266639a46db6b49820b078d160ae1538b020333cd7fe3991734da38e74a4c WatchSource:0}: Error finding container 034266639a46db6b49820b078d160ae1538b020333cd7fe3991734da38e74a4c: Status 404 returned error can't find the container with id 034266639a46db6b49820b078d160ae1538b020333cd7fe3991734da38e74a4c
Sep 10 12:26:22.723624 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:22.723597    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" event={"ID":"7ba4db76-c3f1-482e-954f-d88cbafe472a","Type":"ContainerStarted","Data":"034266639a46db6b49820b078d160ae1538b020333cd7fe3991734da38e74a4c"}
Sep 10 12:26:23.727111 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:23.727078    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" event={"ID":"7ba4db76-c3f1-482e-954f-d88cbafe472a","Type":"ContainerStarted","Data":"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"}
Sep 10 12:26:23.754763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:23.754717    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" podStartSLOduration=1.7547038590000001 podStartE2EDuration="1.754703859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:26:23.753434129 +0000 UTC m=+1011.533991622" watchObservedRunningTime="2026-09-10 12:26:23.754703859 +0000 UTC m=+1011.535261332"
Sep 10 12:26:23.929662 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:23.929635    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:24.848351 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:24.848316    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:24.848782 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:24.848411    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:25.34836931 +0000 UTC m=+1013.128926775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:25.352393 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:25.352344    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:25.352547 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:25.352428    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:26.352415061 +0000 UTC m=+1014.132972512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:26.358616 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:26.358585    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:26.358962 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:26.358645    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:28.358632398 +0000 UTC m=+1016.139189849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:28.373992 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:28.373964    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:28.374441 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:28.374024    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:32.374010952 +0000 UTC m=+1020.154568403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:32.382383 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:32.382353    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:32.408176 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:32.408148    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:32.408303 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:32.408206    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:40.408194576 +0000 UTC m=+1028.188752027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:40.471002 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:40.470970    2551 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-self-signed-certs: secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:40.471360 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:40.471030    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs podName:7ba4db76-c3f1-482e-954f-d88cbafe472a nodeName:}" failed. No retries permitted until 2026-09-10 12:26:56.471017225 +0000 UTC m=+1044.251574677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs") pod "noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a") : secret "noten-model-1d8e56-kserve-self-signed-certs" not found
Sep 10 12:26:40.754937 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:40.754841    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" podUID="7ba4db76-c3f1-482e-954f-d88cbafe472a" containerName="main" containerID="cri-o://7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629" gracePeriod=45
Sep 10 12:26:40.999036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:40.999011    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:41.076457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076400    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076448    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdgjp\" (UniqueName: \"kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076474    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076507    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076588    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076740 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076634    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache\") pod \"7ba4db76-c3f1-482e-954f-d88cbafe472a\" (UID: \"7ba4db76-c3f1-482e-954f-d88cbafe472a\") "
Sep 10 12:26:41.076740 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076640    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home" (OuterVolumeSpecName: "home") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:41.076849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076816    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:41.076933 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076913    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.076988 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076936    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.076988 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.076918    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:41.078661 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.078633    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:41.078774 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.078739    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm" (OuterVolumeSpecName: "dshm") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:26:41.078873 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.078853    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp" (OuterVolumeSpecName: "kube-api-access-pdgjp") pod "7ba4db76-c3f1-482e-954f-d88cbafe472a" (UID: "7ba4db76-c3f1-482e-954f-d88cbafe472a"). InnerVolumeSpecName "kube-api-access-pdgjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:26:41.178032 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.178006    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.178032 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.178025    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdgjp\" (UniqueName: \"kubernetes.io/projected/7ba4db76-c3f1-482e-954f-d88cbafe472a-kube-api-access-pdgjp\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.178032 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.178036    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba4db76-c3f1-482e-954f-d88cbafe472a-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.178196 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.178045    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba4db76-c3f1-482e-954f-d88cbafe472a-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:41.817224 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.817195    2551 generic.go:356] "Generic (PLEG): container finished" podID="7ba4db76-c3f1-482e-954f-d88cbafe472a" containerID="7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629" exitCode=0
Sep 10 12:26:41.817625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.817233    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"
Sep 10 12:26:41.817625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.817262    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" event={"ID":"7ba4db76-c3f1-482e-954f-d88cbafe472a","Type":"ContainerDied","Data":"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"}
Sep 10 12:26:41.817625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.817290    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x" event={"ID":"7ba4db76-c3f1-482e-954f-d88cbafe472a","Type":"ContainerDied","Data":"034266639a46db6b49820b078d160ae1538b020333cd7fe3991734da38e74a4c"}
Sep 10 12:26:41.817625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.817306    2551 scope.go:117] "RemoveContainer" containerID="7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"
Sep 10 12:26:41.826046 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.826029    2551 scope.go:117] "RemoveContainer" containerID="7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"
Sep 10 12:26:41.826316 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:41.826296    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629\": container with ID starting with 7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629 not found: ID does not exist" containerID="7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"
Sep 10 12:26:41.826409 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.826326    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629"} err="failed to get container status \"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629\": rpc error: code = NotFound desc = could not find container \"7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629\": container with ID starting with 7a56212d0205cc762e6b1e730e1dd6b6086cd3e0a251e9e76e75ffa6521cd629 not found: ID does not exist"
Sep 10 12:26:41.843460 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.843431    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:41.848491 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:41.848470    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-1d8e56/noten-model-1d8e56-kserve-7ddddb98bc-bzl6x"]
Sep 10 12:26:42.722694 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:42.722662    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba4db76-c3f1-482e-954f-d88cbafe472a" path="/var/lib/kubelet/pods/7ba4db76-c3f1-482e-954f-d88cbafe472a/volumes"
Sep 10 12:26:53.948995 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:53.948963    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:26:53.949449 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:53.949149    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" podUID="2395444e-d888-4895-80ad-d263d8676f2f" containerName="maas-api" containerID="cri-o://46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c" gracePeriod=30
Sep 10 12:26:54.200705 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.200650    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:26:54.389806 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.389775    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82q5f\" (UniqueName: \"kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f\") pod \"2395444e-d888-4895-80ad-d263d8676f2f\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") "
Sep 10 12:26:54.389966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.389837    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") pod \"2395444e-d888-4895-80ad-d263d8676f2f\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") "
Sep 10 12:26:54.389966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.389869    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca\") pod \"2395444e-d888-4895-80ad-d263d8676f2f\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") "
Sep 10 12:26:54.389966 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.389916    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs\") pod \"2395444e-d888-4895-80ad-d263d8676f2f\" (UID: \"2395444e-d888-4895-80ad-d263d8676f2f\") "
Sep 10 12:26:54.390278 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.390248    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2395444e-d888-4895-80ad-d263d8676f2f" (UID: "2395444e-d888-4895-80ad-d263d8676f2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:26:54.391953 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.391926    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2395444e-d888-4895-80ad-d263d8676f2f" (UID: "2395444e-d888-4895-80ad-d263d8676f2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:54.392059 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.392016    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2395444e-d888-4895-80ad-d263d8676f2f" (UID: "2395444e-d888-4895-80ad-d263d8676f2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:26:54.392246 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.392226    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f" (OuterVolumeSpecName: "kube-api-access-82q5f") pod "2395444e-d888-4895-80ad-d263d8676f2f" (UID: "2395444e-d888-4895-80ad-d263d8676f2f"). InnerVolumeSpecName "kube-api-access-82q5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:26:54.490976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.490917    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82q5f\" (UniqueName: \"kubernetes.io/projected/2395444e-d888-4895-80ad-d263d8676f2f-kube-api-access-82q5f\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:54.490976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.490941    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:54.498045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.491350    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2395444e-d888-4895-80ad-d263d8676f2f-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:54.498045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.491395    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2395444e-d888-4895-80ad-d263d8676f2f-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:26:54.882549 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.882517    2551 generic.go:356] "Generic (PLEG): container finished" podID="2395444e-d888-4895-80ad-d263d8676f2f" containerID="46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c" exitCode=0
Sep 10 12:26:54.882735 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.882585    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"
Sep 10 12:26:54.882735 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.882683    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" event={"ID":"2395444e-d888-4895-80ad-d263d8676f2f","Type":"ContainerDied","Data":"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"}
Sep 10 12:26:54.882735 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.882732    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq" event={"ID":"2395444e-d888-4895-80ad-d263d8676f2f","Type":"ContainerDied","Data":"82089df8db6050f93454def6b7732b0a6859f58636f52a36a267afccfe999028"}
Sep 10 12:26:54.882896 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.882759    2551 scope.go:117] "RemoveContainer" containerID="46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"
Sep 10 12:26:54.891714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.891683    2551 scope.go:117] "RemoveContainer" containerID="46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"
Sep 10 12:26:54.891963 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:54.891943    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c\": container with ID starting with 46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c not found: ID does not exist" containerID="46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"
Sep 10 12:26:54.892006 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.891970    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c"} err="failed to get container status \"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c\": rpc error: code = NotFound desc = could not find container \"46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c\": container with ID starting with 46c0b29ef2c42342a4314ebc75533a7b4d6a50ce5fbe80c5b25c551627798a3c not found: ID does not exist"
Sep 10 12:26:54.944940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.944917    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:26:54.958521 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:54.958498    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7576a1-5b8f84d8b9-tmhzq"]
Sep 10 12:26:56.723207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:56.723167    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2395444e-d888-4895-80ad-d263d8676f2f" path="/var/lib/kubelet/pods/2395444e-d888-4895-80ad-d263d8676f2f/volumes"
Sep 10 12:26:57.663902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.663860    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:26:57.664499 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664479    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba4db76-c3f1-482e-954f-d88cbafe472a" containerName="main"
Sep 10 12:26:57.664566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664502    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba4db76-c3f1-482e-954f-d88cbafe472a" containerName="main"
Sep 10 12:26:57.664566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664528    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2395444e-d888-4895-80ad-d263d8676f2f" containerName="maas-api"
Sep 10 12:26:57.664566 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664537    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2395444e-d888-4895-80ad-d263d8676f2f" containerName="maas-api"
Sep 10 12:26:57.664674 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664662    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba4db76-c3f1-482e-954f-d88cbafe472a" containerName="main"
Sep 10 12:26:57.664718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.664676    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2395444e-d888-4895-80ad-d263d8676f2f" containerName="maas-api"
Sep 10 12:26:57.669678 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.669658    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.673324 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.673300    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-062bb857-serving-cert\""
Sep 10 12:26:57.690527 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.684850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:26:57.718418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.718369    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.718539 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.718487    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.718539 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.718514    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.718654 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.718563    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6mc7\" (UniqueName: \"kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.819262 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.819231    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.819670 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.819302    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.819670 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.819340    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.819670 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.819427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6mc7\" (UniqueName: \"kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.819670 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:57.819474    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-serving-cert: secret "maas-api-e2e-mt-062bb857-serving-cert" not found
Sep 10 12:26:57.819670 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:57.819580    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls podName:cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7 nodeName:}" failed. No retries permitted until 2026-09-10 12:26:58.319558958 +0000 UTC m=+1046.100116411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls") pod "maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7") : secret "maas-api-e2e-mt-062bb857-serving-cert" not found
Sep 10 12:26:57.820016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.819990    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.821972 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.821945    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:57.834255 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:57.834230    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6mc7\" (UniqueName: \"kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:58.322801 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:58.322764    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:58.322967 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:58.322916    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-serving-cert: secret "maas-api-e2e-mt-062bb857-serving-cert" not found
Sep 10 12:26:58.323011 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:26:58.322979    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls podName:cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7 nodeName:}" failed. No retries permitted until 2026-09-10 12:26:59.322964209 +0000 UTC m=+1047.103521665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls") pod "maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7") : secret "maas-api-e2e-mt-062bb857-serving-cert" not found
Sep 10 12:26:59.331876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.331844    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:59.334008 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.333984    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:59.481047 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.481016    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:26:59.760242 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.760219    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:26:59.762324 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:26:59.762298    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc69b2f_e56f_49b5_bb12_ca2ada39a4d7.slice/crio-4aff7c62f38bd2d964fea541e3082a51ad247bb7c4a6e217fc18cf99a5ee51fa WatchSource:0}: Error finding container 4aff7c62f38bd2d964fea541e3082a51ad247bb7c4a6e217fc18cf99a5ee51fa: Status 404 returned error can't find the container with id 4aff7c62f38bd2d964fea541e3082a51ad247bb7c4a6e217fc18cf99a5ee51fa
Sep 10 12:26:59.764090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.764074    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:26:59.909349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:26:59.909320    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" event={"ID":"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7","Type":"ContainerStarted","Data":"4aff7c62f38bd2d964fea541e3082a51ad247bb7c4a6e217fc18cf99a5ee51fa"}
Sep 10 12:27:00.922544 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:00.922506    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" event={"ID":"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7","Type":"ContainerStarted","Data":"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"}
Sep 10 12:27:00.922875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:00.922637    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:27:00.947637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:00.947595    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" podStartSLOduration=3.511731648 podStartE2EDuration="3.947583428s" totalImagesPullingTime="435.85178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:27:00.944650596 +0000 UTC m=+1048.725208069" watchObservedRunningTime="2026-09-10 12:27:00.947583428 +0000 UTC m=+1048.728140901"
Sep 10 12:27:06.934525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:06.934496    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:27:12.613792 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.613760    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:27:12.614165 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.613954    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" podUID="c815a526-83d1-4f41-846d-324623f38aa2" containerName="maas-api" containerID="cri-o://67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c" gracePeriod=30
Sep 10 12:27:12.864988 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.864931    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:27:12.940002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.939975    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs\") pod \"c815a526-83d1-4f41-846d-324623f38aa2\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") "
Sep 10 12:27:12.940002 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.940010    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca\") pod \"c815a526-83d1-4f41-846d-324623f38aa2\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") "
Sep 10 12:27:12.940239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.940029    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls\") pod \"c815a526-83d1-4f41-846d-324623f38aa2\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") "
Sep 10 12:27:12.940239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.940066    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65qm5\" (UniqueName: \"kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5\") pod \"c815a526-83d1-4f41-846d-324623f38aa2\" (UID: \"c815a526-83d1-4f41-846d-324623f38aa2\") "
Sep 10 12:27:12.940679 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.940651    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c815a526-83d1-4f41-846d-324623f38aa2" (UID: "c815a526-83d1-4f41-846d-324623f38aa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:27:12.942358 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.942321    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5" (OuterVolumeSpecName: "kube-api-access-65qm5") pod "c815a526-83d1-4f41-846d-324623f38aa2" (UID: "c815a526-83d1-4f41-846d-324623f38aa2"). InnerVolumeSpecName "kube-api-access-65qm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:27:12.942592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.942565    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c815a526-83d1-4f41-846d-324623f38aa2" (UID: "c815a526-83d1-4f41-846d-324623f38aa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:27:12.942747 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.942721    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c815a526-83d1-4f41-846d-324623f38aa2" (UID: "c815a526-83d1-4f41-846d-324623f38aa2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:27:12.989891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.989861    2551 generic.go:356] "Generic (PLEG): container finished" podID="c815a526-83d1-4f41-846d-324623f38aa2" containerID="67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c" exitCode=0
Sep 10 12:27:12.990038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.989919    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"
Sep 10 12:27:12.990038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.989949    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" event={"ID":"c815a526-83d1-4f41-846d-324623f38aa2","Type":"ContainerDied","Data":"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"}
Sep 10 12:27:12.990038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.990017    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk" event={"ID":"c815a526-83d1-4f41-846d-324623f38aa2","Type":"ContainerDied","Data":"7f3906185c92437c1b8598defa9f2f4ca1b0def2b38261b8ce23219255c61907"}
Sep 10 12:27:12.990162 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.990039    2551 scope.go:117] "RemoveContainer" containerID="67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"
Sep 10 12:27:12.998911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.998893    2551 scope.go:117] "RemoveContainer" containerID="67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"
Sep 10 12:27:12.999171 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:12.999150    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c\": container with ID starting with 67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c not found: ID does not exist" containerID="67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"
Sep 10 12:27:12.999239 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:12.999182    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c"} err="failed to get container status \"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c\": rpc error: code = NotFound desc = could not find container \"67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c\": container with ID starting with 67da090069d41378df942fcfdf2b266759785e9a7d50900e28c0ea35e45d597c not found: ID does not exist"
Sep 10 12:27:13.014465 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.014435    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:27:13.016448 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.016430    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb35b9-754d6875bb-jqnbk"]
Sep 10 12:27:13.040858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.040844    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:13.040925 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.040861    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c815a526-83d1-4f41-846d-324623f38aa2-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:13.040925 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.040870    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c815a526-83d1-4f41-846d-324623f38aa2-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:13.040925 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:13.040878    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65qm5\" (UniqueName: \"kubernetes.io/projected/c815a526-83d1-4f41-846d-324623f38aa2-kube-api-access-65qm5\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:14.724242 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:14.724205    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c815a526-83d1-4f41-846d-324623f38aa2" path="/var/lib/kubelet/pods/c815a526-83d1-4f41-846d-324623f38aa2/volumes"
Sep 10 12:27:25.994548 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:25.994470    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:27:25.995073 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:25.995053    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c815a526-83d1-4f41-846d-324623f38aa2" containerName="maas-api"
Sep 10 12:27:25.995149 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:25.995076    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c815a526-83d1-4f41-846d-324623f38aa2" containerName="maas-api"
Sep 10 12:27:25.995224 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:25.995203    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c815a526-83d1-4f41-846d-324623f38aa2" containerName="maas-api"
Sep 10 12:27:25.999959 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:25.999938    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.003860 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.003843    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-18c2d2-serving-cert\""
Sep 10 12:27:26.041322 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.041298    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:27:26.046428 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.046407    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.046531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.046455    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.046531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.046482    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.046531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.046518    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2wdt\" (UniqueName: \"kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.147430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.147407    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.147533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.147437    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.147533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.147463    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2wdt\" (UniqueName: \"kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.147666 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.147559    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.147709 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:26.147680    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-serving-cert: secret "maas-api-e2e-shared-a-w3-18c2d2-serving-cert" not found
Sep 10 12:27:26.147785 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:26.147767    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls podName:c354b2d2-723a-41ba-9322-6fea1210cfc4 nodeName:}" failed. No retries permitted until 2026-09-10 12:27:26.647748119 +0000 UTC m=+1074.428305570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls") pod "maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" (UID: "c354b2d2-723a-41ba-9322-6fea1210cfc4") : secret "maas-api-e2e-shared-a-w3-18c2d2-serving-cert" not found
Sep 10 12:27:26.148117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.148098    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.149847 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.149824    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.169591 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.169569    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2wdt\" (UniqueName: \"kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.651090 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.651059    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.653310 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.653281    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:26.912119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:26.912038    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:27.197267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:27.197243    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:27:27.200155 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:27:27.200129    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc354b2d2_723a_41ba_9322_6fea1210cfc4.slice/crio-a5c14c488dca69e6ed1c2bc0c2c19beef36a7cc418f4b0d0f2361b394abea2a7 WatchSource:0}: Error finding container a5c14c488dca69e6ed1c2bc0c2c19beef36a7cc418f4b0d0f2361b394abea2a7: Status 404 returned error can't find the container with id a5c14c488dca69e6ed1c2bc0c2c19beef36a7cc418f4b0d0f2361b394abea2a7
Sep 10 12:27:28.071730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:28.071696    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" event={"ID":"c354b2d2-723a-41ba-9322-6fea1210cfc4","Type":"ContainerStarted","Data":"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"}
Sep 10 12:27:28.071903 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:28.071744    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" event={"ID":"c354b2d2-723a-41ba-9322-6fea1210cfc4","Type":"ContainerStarted","Data":"a5c14c488dca69e6ed1c2bc0c2c19beef36a7cc418f4b0d0f2361b394abea2a7"}
Sep 10 12:27:28.071903 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:28.071839    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:28.095226 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:28.095169    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" podStartSLOduration=2.768444289 podStartE2EDuration="3.095156152s" totalImagesPullingTime="326.711863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:27:28.093157408 +0000 UTC m=+1075.873714883" watchObservedRunningTime="2026-09-10 12:27:28.095156152 +0000 UTC m=+1075.875713667"
Sep 10 12:27:34.081535 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:34.081502    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:27:40.766496 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:40.766463    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:27:40.766945 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:40.766668    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" podUID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" containerName="maas-api" containerID="cri-o://0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c" gracePeriod=30
Sep 10 12:27:41.025105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.025058    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:27:41.142170 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.142142    2551 generic.go:356] "Generic (PLEG): container finished" podID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" containerID="0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c" exitCode=0
Sep 10 12:27:41.142301 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.142180    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"
Sep 10 12:27:41.142301 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.142234    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" event={"ID":"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7","Type":"ContainerDied","Data":"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"}
Sep 10 12:27:41.142301 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.142286    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x" event={"ID":"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7","Type":"ContainerDied","Data":"4aff7c62f38bd2d964fea541e3082a51ad247bb7c4a6e217fc18cf99a5ee51fa"}
Sep 10 12:27:41.142438 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.142312    2551 scope.go:117] "RemoveContainer" containerID="0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"
Sep 10 12:27:41.151118 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.151102    2551 scope.go:117] "RemoveContainer" containerID="0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"
Sep 10 12:27:41.151396 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:41.151357    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c\": container with ID starting with 0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c not found: ID does not exist" containerID="0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"
Sep 10 12:27:41.151476 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.151409    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c"} err="failed to get container status \"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c\": rpc error: code = NotFound desc = could not find container \"0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c\": container with ID starting with 0ddee9bbd844fcdcfc022b55df6bc514f73304014d92188c3a3624ede258748c not found: ID does not exist"
Sep 10 12:27:41.167130 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167110    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs\") pod \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") "
Sep 10 12:27:41.167219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167173    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") pod \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") "
Sep 10 12:27:41.167282 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167266    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca\") pod \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") "
Sep 10 12:27:41.167362 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167349    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6mc7\" (UniqueName: \"kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7\") pod \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\" (UID: \"cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7\") "
Sep 10 12:27:41.167619 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167597    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:27:41.167717 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.167698    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:41.169207 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.169176    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:27:41.169306 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.169257    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:27:41.169367 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.169311    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7" (OuterVolumeSpecName: "kube-api-access-x6mc7") pod "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" (UID: "cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7"). InnerVolumeSpecName "kube-api-access-x6mc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:27:41.268113 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.268092    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6mc7\" (UniqueName: \"kubernetes.io/projected/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-kube-api-access-x6mc7\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:41.268203 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.268114    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:41.268203 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.268128    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:27:41.466511 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.466490    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:27:41.468274 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:41.468252    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-062bb857-bc5f76c6-kzs4x"]
Sep 10 12:27:42.722911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:42.722881    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" path="/var/lib/kubelet/pods/cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7/volumes"
Sep 10 12:27:54.347920 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.347886    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:27:54.348354 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.348337    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" containerName="maas-api"
Sep 10 12:27:54.348416 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.348357    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" containerName="maas-api"
Sep 10 12:27:54.348464 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.348457    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbc69b2f-e56f-49b5-bb12-ca2ada39a4d7" containerName="maas-api"
Sep 10 12:27:54.352886 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.352869    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.361952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.361930    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-f89a79-serving-cert\""
Sep 10 12:27:54.370784 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.370763    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:27:54.468943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.468911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxp6\" (UniqueName: \"kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.469076 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.468985    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.469076 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.469050    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.469167 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.469081    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.570529 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.570493    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.570709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.570546    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.570709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.570564    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.570709 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.570626    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxp6\" (UniqueName: \"kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.570709 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:54.570699    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-serving-cert: secret "maas-api-e2e-shared-b-w3-f89a79-serving-cert" not found
Sep 10 12:27:54.570915 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:27:54.570767    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls podName:a2b1b76a-cf06-4771-9588-fa33eef78a54 nodeName:}" failed. No retries permitted until 2026-09-10 12:27:55.070750652 +0000 UTC m=+1102.851308106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls") pod "maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" (UID: "a2b1b76a-cf06-4771-9588-fa33eef78a54") : secret "maas-api-e2e-shared-b-w3-f89a79-serving-cert" not found
Sep 10 12:27:54.571317 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.571300    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.573038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.573018    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:54.584876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:54.584848    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxp6\" (UniqueName: \"kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:55.075045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:55.075013    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:55.077313 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:55.077291    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:55.265232 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:55.265204    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:55.562336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:55.562311    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:27:55.563825 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:27:55.563797    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b1b76a_cf06_4771_9588_fa33eef78a54.slice/crio-08713d9f36f822bdb4029f91a55b065691407db3c8e8b757864d1da5e24f501b WatchSource:0}: Error finding container 08713d9f36f822bdb4029f91a55b065691407db3c8e8b757864d1da5e24f501b: Status 404 returned error can't find the container with id 08713d9f36f822bdb4029f91a55b065691407db3c8e8b757864d1da5e24f501b
Sep 10 12:27:56.226480 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:56.226403    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" event={"ID":"a2b1b76a-cf06-4771-9588-fa33eef78a54","Type":"ContainerStarted","Data":"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"}
Sep 10 12:27:56.226480 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:56.226454    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" event={"ID":"a2b1b76a-cf06-4771-9588-fa33eef78a54","Type":"ContainerStarted","Data":"08713d9f36f822bdb4029f91a55b065691407db3c8e8b757864d1da5e24f501b"}
Sep 10 12:27:56.226637 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:56.226504    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:27:56.268081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:27:56.268038    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" podStartSLOduration=1.88329717 podStartE2EDuration="2.268026318s" totalImagesPullingTime="384.729148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:27:56.267747277 +0000 UTC m=+1104.048304764" watchObservedRunningTime="2026-09-10 12:27:56.268026318 +0000 UTC m=+1104.048583825"
Sep 10 12:28:02.236952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:02.236921    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:28:09.164118 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.164086    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:28:09.167837 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.167803    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.170461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.170434    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-d1e553-serving-cert\""
Sep 10 12:28:09.180043 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.180018    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:28:09.192267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.192246    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.192418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.192274    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.192418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.192296    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.192418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.192318    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmnh\" (UniqueName: \"kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.292964 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.292935    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.293148 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.292969    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.293148 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.292998    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.293148 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.293033    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmnh\" (UniqueName: \"kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.293148 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:28:09.293125    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-serving-cert: secret "maas-api-e2e-shared-a-w2-d1e553-serving-cert" not found
Sep 10 12:28:09.293426 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:28:09.293204    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls podName:7c6c2880-9b37-4467-b06e-82ad1e9a601c nodeName:}" failed. No retries permitted until 2026-09-10 12:28:09.793183426 +0000 UTC m=+1117.573740884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls") pod "maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" (UID: "7c6c2880-9b37-4467-b06e-82ad1e9a601c") : secret "maas-api-e2e-shared-a-w2-d1e553-serving-cert" not found
Sep 10 12:28:09.293765 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.293746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.295553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.295526    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.304674 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.304647    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmnh\" (UniqueName: \"kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.798056 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.798022    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:09.800262 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:09.800240    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:10.080855 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:10.080831    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:10.353734 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:10.353705    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:28:10.355873 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:28:10.355846    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6c2880_9b37_4467_b06e_82ad1e9a601c.slice/crio-772fb93b39cd0dd686aff59757cc7e19ceb84bff3704d88bacd1d905de734d0b WatchSource:0}: Error finding container 772fb93b39cd0dd686aff59757cc7e19ceb84bff3704d88bacd1d905de734d0b: Status 404 returned error can't find the container with id 772fb93b39cd0dd686aff59757cc7e19ceb84bff3704d88bacd1d905de734d0b
Sep 10 12:28:11.307577 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:11.307541    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" event={"ID":"7c6c2880-9b37-4467-b06e-82ad1e9a601c","Type":"ContainerStarted","Data":"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"}
Sep 10 12:28:11.307750 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:11.307631    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" event={"ID":"7c6c2880-9b37-4467-b06e-82ad1e9a601c","Type":"ContainerStarted","Data":"772fb93b39cd0dd686aff59757cc7e19ceb84bff3704d88bacd1d905de734d0b"}
Sep 10 12:28:11.307858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:11.307749    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:17.318094 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:17.318062    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:28:17.345036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:17.344981    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" podStartSLOduration=7.923058646 podStartE2EDuration="8.344966544s" totalImagesPullingTime="421.907898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:28:11.330728901 +0000 UTC m=+1119.111286393" watchObservedRunningTime="2026-09-10 12:28:17.344966544 +0000 UTC m=+1125.125524017"
Sep 10 12:28:37.580369 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.580332    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:28:37.585411 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.585393    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.590550 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.590520    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-a08ea2-serving-cert\""
Sep 10 12:28:37.598046 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.598023    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:28:37.719053 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.719026    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.719219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.719067    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwngs\" (UniqueName: \"kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.719219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.719090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.719219 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.719159    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.820615 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.820581    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.820749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.820696    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwngs\" (UniqueName: \"kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.820895 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.820851    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.821119 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.821098    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.821272 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:28:37.821153    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-serving-cert: secret "maas-api-e2e-shared-b-w2-a08ea2-serving-cert" not found
Sep 10 12:28:37.821347 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:28:37.821337    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls podName:43abb84a-421b-486b-b1d6-9165b361ee72 nodeName:}" failed. No retries permitted until 2026-09-10 12:28:38.321317706 +0000 UTC m=+1146.101875164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls") pod "maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" (UID: "43abb84a-421b-486b-b1d6-9165b361ee72") : secret "maas-api-e2e-shared-b-w2-a08ea2-serving-cert" not found
Sep 10 12:28:37.821430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.821353    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.823496 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.823475    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:37.837027 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:37.836966    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwngs\" (UniqueName: \"kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:38.327158 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:38.327064    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:38.329871 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:38.329845    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:38.499786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:38.499758    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:38.782430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:38.782346    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:28:38.786524 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:28:38.786493    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43abb84a_421b_486b_b1d6_9165b361ee72.slice/crio-98a6197f5fc51852bdd47f035254d20f2f29875ba363509677a1f790c0f084de WatchSource:0}: Error finding container 98a6197f5fc51852bdd47f035254d20f2f29875ba363509677a1f790c0f084de: Status 404 returned error can't find the container with id 98a6197f5fc51852bdd47f035254d20f2f29875ba363509677a1f790c0f084de
Sep 10 12:28:39.466797 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:39.466690    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" event={"ID":"43abb84a-421b-486b-b1d6-9165b361ee72","Type":"ContainerStarted","Data":"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"}
Sep 10 12:28:39.466951 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:39.466803    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" event={"ID":"43abb84a-421b-486b-b1d6-9165b361ee72","Type":"ContainerStarted","Data":"98a6197f5fc51852bdd47f035254d20f2f29875ba363509677a1f790c0f084de"}
Sep 10 12:28:39.466951 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:39.466852    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:28:39.489974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:39.489923    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" podStartSLOduration=2.065139673 podStartE2EDuration="2.489906819s" totalImagesPullingTime="424.767146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:28:39.487099445 +0000 UTC m=+1147.267656937" watchObservedRunningTime="2026-09-10 12:28:39.489906819 +0000 UTC m=+1147.270464292"
Sep 10 12:28:45.481977 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:28:45.481942    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:29:09.068275 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.068199    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:29:09.071739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.071723    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.075606 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.075587    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f89a79\"/\"kube-root-ca.crt\""
Sep 10 12:29:09.075734 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.075711    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f89a79\"/\"openshift-service-ca.crt\""
Sep 10 12:29:09.075968 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.075953    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f89a79\"/\"auth-test-model-f89a79-kserve-self-signed-certs\""
Sep 10 12:29:09.076042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.075957    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f89a79\"/\"default-dockercfg-rfdfk\""
Sep 10 12:29:09.088902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.088878    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:29:09.174510 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174487    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.174629 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174541    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.174676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174627    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9gr9\" (UniqueName: \"kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.174676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174658    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.174794 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174697    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.174794 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.174741    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276174 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276154    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276302 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276193    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276302 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276209    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276302 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276238    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276302 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276268    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276302 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276300    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gr9\" (UniqueName: \"kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276638 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276618    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276710 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276654    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.276710 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.276688    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.278351 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.278329    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.278687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.278671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.285863 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.285843    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9gr9\" (UniqueName: \"kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9\") pod \"auth-test-model-f89a79-kserve-6dc588995f-whk5w\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") " pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.381723 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.381702    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:09.655809 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:09.655784    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:29:09.657575 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:29:09.657542    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15d52e58_5318_47b8_821f_136041cb12f3.slice/crio-c834d500bdaefcaef1a57ce5e29061ff1b10171da81fce77a310061d4f0ee777 WatchSource:0}: Error finding container c834d500bdaefcaef1a57ce5e29061ff1b10171da81fce77a310061d4f0ee777: Status 404 returned error can't find the container with id c834d500bdaefcaef1a57ce5e29061ff1b10171da81fce77a310061d4f0ee777
Sep 10 12:29:10.635488 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:10.635452    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" event={"ID":"15d52e58-5318-47b8-821f-136041cb12f3","Type":"ContainerStarted","Data":"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"}
Sep 10 12:29:10.635488 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:10.635493    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" event={"ID":"15d52e58-5318-47b8-821f-136041cb12f3","Type":"ContainerStarted","Data":"c834d500bdaefcaef1a57ce5e29061ff1b10171da81fce77a310061d4f0ee777"}
Sep 10 12:29:10.662669 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:10.662625    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" podStartSLOduration=1.662610541 podStartE2EDuration="1.662610541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:10.662279874 +0000 UTC m=+1178.442837345" watchObservedRunningTime="2026-09-10 12:29:10.662610541 +0000 UTC m=+1178.443168013"
Sep 10 12:29:18.771332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.771278    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:29:18.777724 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.777699    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.780168 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.780143    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-156d6252-serving-cert\""
Sep 10 12:29:18.791161 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.791140    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:29:18.862851 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.862815    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.863001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.862962    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.863001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.862995    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.863117 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.863034    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6gd4\" (UniqueName: \"kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.963791 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.963761    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.963939 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.963824    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.963939 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.963842    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.963939 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.963867    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6gd4\" (UniqueName: \"kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.964501 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.964468    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.966237 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.966217    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.966299 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.966224    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:18.973603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:18.973581    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6gd4\" (UniqueName: \"kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4\") pod \"maas-api-e2e-mt-156d6252-7c577475c7-fn2q5\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:19.089560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.089539    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:19.364523 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.364494    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:29:19.365995 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:29:19.365967    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d9ca79_5e0b_48ad_992c_444a4d0db357.slice/crio-b1f8674cb68fe3b1cca7113c1058b0ffb70435124b89fda7ea5582b31767ce09 WatchSource:0}: Error finding container b1f8674cb68fe3b1cca7113c1058b0ffb70435124b89fda7ea5582b31767ce09: Status 404 returned error can't find the container with id b1f8674cb68fe3b1cca7113c1058b0ffb70435124b89fda7ea5582b31767ce09
Sep 10 12:29:19.381829 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.381806    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:19.381946 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.381850    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:19.394224 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.394205    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:19.683478 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.683403    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" event={"ID":"47d9ca79-5e0b-48ad-992c-444a4d0db357","Type":"ContainerStarted","Data":"b1f8674cb68fe3b1cca7113c1058b0ffb70435124b89fda7ea5582b31767ce09"}
Sep 10 12:29:19.694920 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:19.694896    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:29:20.690214 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:20.690180    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" event={"ID":"47d9ca79-5e0b-48ad-992c-444a4d0db357","Type":"ContainerStarted","Data":"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"}
Sep 10 12:29:20.710473 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:20.710431    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" podStartSLOduration=2.120174809 podStartE2EDuration="2.710417407s" totalImagesPullingTime="590.242598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:20.708582834 +0000 UTC m=+1188.489140306" watchObservedRunningTime="2026-09-10 12:29:20.710417407 +0000 UTC m=+1188.490974890"
Sep 10 12:29:21.696774 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:21.696742    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:27.706177 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:27.706149    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:29:32.708984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:32.708959    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:29:32.711579 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:32.711555    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:29:32.713368 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:32.713349    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:29:32.722898 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:29:32.722867    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:30:42.268466 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.268396    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:30:42.268977 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.268599    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" podUID="47d9ca79-5e0b-48ad-992c-444a4d0db357" containerName="maas-api" containerID="cri-o://986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578" gracePeriod=30
Sep 10 12:30:42.516069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.516046    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:30:42.663849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.663822    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca\") pod \"47d9ca79-5e0b-48ad-992c-444a4d0db357\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") "
Sep 10 12:30:42.664010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.663857    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs\") pod \"47d9ca79-5e0b-48ad-992c-444a4d0db357\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") "
Sep 10 12:30:42.664010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.663880    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls\") pod \"47d9ca79-5e0b-48ad-992c-444a4d0db357\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") "
Sep 10 12:30:42.664010 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.663939    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6gd4\" (UniqueName: \"kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4\") pod \"47d9ca79-5e0b-48ad-992c-444a4d0db357\" (UID: \"47d9ca79-5e0b-48ad-992c-444a4d0db357\") "
Sep 10 12:30:42.664248 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.664222    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47d9ca79-5e0b-48ad-992c-444a4d0db357" (UID: "47d9ca79-5e0b-48ad-992c-444a4d0db357"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:30:42.666052 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.666021    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4" (OuterVolumeSpecName: "kube-api-access-f6gd4") pod "47d9ca79-5e0b-48ad-992c-444a4d0db357" (UID: "47d9ca79-5e0b-48ad-992c-444a4d0db357"). InnerVolumeSpecName "kube-api-access-f6gd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:30:42.666440 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.666411    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47d9ca79-5e0b-48ad-992c-444a4d0db357" (UID: "47d9ca79-5e0b-48ad-992c-444a4d0db357"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:30:42.666440 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.666422    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "47d9ca79-5e0b-48ad-992c-444a4d0db357" (UID: "47d9ca79-5e0b-48ad-992c-444a4d0db357"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:30:42.765603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.765577    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d9ca79-5e0b-48ad-992c-444a4d0db357-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:30:42.765603 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.765602    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:30:42.765772 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.765611    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47d9ca79-5e0b-48ad-992c-444a4d0db357-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:30:42.765772 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:42.765618    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6gd4\" (UniqueName: \"kubernetes.io/projected/47d9ca79-5e0b-48ad-992c-444a4d0db357-kube-api-access-f6gd4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:30:43.125814 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.125783    2551 generic.go:356] "Generic (PLEG): container finished" podID="47d9ca79-5e0b-48ad-992c-444a4d0db357" containerID="986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578" exitCode=0
Sep 10 12:30:43.125957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.125830    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"
Sep 10 12:30:43.125957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.125874    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" event={"ID":"47d9ca79-5e0b-48ad-992c-444a4d0db357","Type":"ContainerDied","Data":"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"}
Sep 10 12:30:43.125957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.125925    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5" event={"ID":"47d9ca79-5e0b-48ad-992c-444a4d0db357","Type":"ContainerDied","Data":"b1f8674cb68fe3b1cca7113c1058b0ffb70435124b89fda7ea5582b31767ce09"}
Sep 10 12:30:43.125957 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.125950    2551 scope.go:117] "RemoveContainer" containerID="986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"
Sep 10 12:30:43.134691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.134673    2551 scope.go:117] "RemoveContainer" containerID="986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"
Sep 10 12:30:43.134945 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:30:43.134926    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578\": container with ID starting with 986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578 not found: ID does not exist" containerID="986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"
Sep 10 12:30:43.135013 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.134956    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578"} err="failed to get container status \"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578\": rpc error: code = NotFound desc = could not find container \"986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578\": container with ID starting with 986885ccc1e007098d97780399cea758d45e5008d1bd5dc70631e4ae5a011578 not found: ID does not exist"
Sep 10 12:30:43.148984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.148962    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:30:43.156652 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:43.156631    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-156d6252-7c577475c7-fn2q5"]
Sep 10 12:30:44.726292 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:30:44.726258    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d9ca79-5e0b-48ad-992c-444a4d0db357" path="/var/lib/kubelet/pods/47d9ca79-5e0b-48ad-992c-444a4d0db357/volumes"
Sep 10 12:31:10.664999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:10.664960    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:31:25.684583 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:25.684536    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" podUID="15d52e58-5318-47b8-821f-136041cb12f3" containerName="main" containerID="cri-o://e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c" gracePeriod=45
Sep 10 12:31:25.933102 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:25.933079    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:31:26.047621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047550    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.047621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047591    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9gr9\" (UniqueName: \"kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.047621 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047617    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.047876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047665    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.047876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047698    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.047876 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047733    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs\") pod \"15d52e58-5318-47b8-821f-136041cb12f3\" (UID: \"15d52e58-5318-47b8-821f-136041cb12f3\") "
Sep 10 12:31:26.048034 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047921    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home" (OuterVolumeSpecName: "home") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:31:26.048034 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047955    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:31:26.048034 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.047984    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:31:26.048180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.048117    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.048180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.048135    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.048180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.048173    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.049786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.049763    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9" (OuterVolumeSpecName: "kube-api-access-n9gr9") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "kube-api-access-n9gr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:31:26.049786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.049770    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm" (OuterVolumeSpecName: "dshm") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:31:26.049953 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.049830    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15d52e58-5318-47b8-821f-136041cb12f3" (UID: "15d52e58-5318-47b8-821f-136041cb12f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:31:26.149406 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.149362    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d52e58-5318-47b8-821f-136041cb12f3-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.149406 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.149403    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9gr9\" (UniqueName: \"kubernetes.io/projected/15d52e58-5318-47b8-821f-136041cb12f3-kube-api-access-n9gr9\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.149571 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.149414    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d52e58-5318-47b8-821f-136041cb12f3-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:31:26.352402 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.352351    2551 generic.go:356] "Generic (PLEG): container finished" podID="15d52e58-5318-47b8-821f-136041cb12f3" containerID="e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c" exitCode=0
Sep 10 12:31:26.352559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.352415    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"
Sep 10 12:31:26.352559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.352412    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" event={"ID":"15d52e58-5318-47b8-821f-136041cb12f3","Type":"ContainerDied","Data":"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"}
Sep 10 12:31:26.352559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.352456    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w" event={"ID":"15d52e58-5318-47b8-821f-136041cb12f3","Type":"ContainerDied","Data":"c834d500bdaefcaef1a57ce5e29061ff1b10171da81fce77a310061d4f0ee777"}
Sep 10 12:31:26.352559 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.352475    2551 scope.go:117] "RemoveContainer" containerID="e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"
Sep 10 12:31:26.361701 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.361685    2551 scope.go:117] "RemoveContainer" containerID="e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"
Sep 10 12:31:26.361959 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:31:26.361940    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c\": container with ID starting with e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c not found: ID does not exist" containerID="e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"
Sep 10 12:31:26.362025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.361966    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c"} err="failed to get container status \"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c\": rpc error: code = NotFound desc = could not find container \"e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c\": container with ID starting with e69cd43708b91267e2a9bda347be18e8b8847146edc1d74800b89e7bdcb1d84c not found: ID does not exist"
Sep 10 12:31:26.396337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.396313    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:31:26.403995 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.403974    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f89a79/auth-test-model-f89a79-kserve-6dc588995f-whk5w"]
Sep 10 12:31:26.723298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:31:26.723207    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d52e58-5318-47b8-821f-136041cb12f3" path="/var/lib/kubelet/pods/15d52e58-5318-47b8-821f-136041cb12f3/volumes"
Sep 10 12:32:35.792641 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.792607    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:32:35.793227 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793203    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d9ca79-5e0b-48ad-992c-444a4d0db357" containerName="maas-api"
Sep 10 12:32:35.793227 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793223    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d9ca79-5e0b-48ad-992c-444a4d0db357" containerName="maas-api"
Sep 10 12:32:35.793446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793236    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d52e58-5318-47b8-821f-136041cb12f3" containerName="main"
Sep 10 12:32:35.793446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793244    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d52e58-5318-47b8-821f-136041cb12f3" containerName="main"
Sep 10 12:32:35.793446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793353    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d52e58-5318-47b8-821f-136041cb12f3" containerName="main"
Sep 10 12:32:35.793446 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.793384    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d9ca79-5e0b-48ad-992c-444a4d0db357" containerName="maas-api"
Sep 10 12:32:35.797077 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.797058    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.800695 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.800677    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5ff68c-serving-cert\""
Sep 10 12:32:35.816809 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.816787    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:32:35.881160 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.881126    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.881306 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.881192    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.881306 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.881273    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld956\" (UniqueName: \"kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.881395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.881355    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.982471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.982437    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.982587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.982489    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.982587 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.982532    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.982672 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.982607    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld956\" (UniqueName: \"kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.982731 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:32:35.982712    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-serving-cert: secret "maas-api-e2e-api-a-5ff68c-serving-cert" not found
Sep 10 12:32:35.982809 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:32:35.982797    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls podName:35c4c1a4-0a23-41f2-afe1-8f9a613f77fb nodeName:}" failed. No retries permitted until 2026-09-10 12:32:36.482777436 +0000 UTC m=+1384.263334904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls") pod "maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" (UID: "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb") : secret "maas-api-e2e-api-a-5ff68c-serving-cert" not found
Sep 10 12:32:35.983360 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.983339    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.984850 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.984833    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:35.999143 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:35.999114    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld956\" (UniqueName: \"kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:36.486471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:36.486439    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:36.488729 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:36.488705    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") pod \"maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:36.708642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:36.708610    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:36.984730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:36.984701    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:32:36.991960 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:32:36.991935    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c4c1a4_0a23_41f2_afe1_8f9a613f77fb.slice/crio-8832656c562e7014b271db90f5e6d9b0209e3c396633939f3fd2a07e1efb5374 WatchSource:0}: Error finding container 8832656c562e7014b271db90f5e6d9b0209e3c396633939f3fd2a07e1efb5374: Status 404 returned error can't find the container with id 8832656c562e7014b271db90f5e6d9b0209e3c396633939f3fd2a07e1efb5374
Sep 10 12:32:36.993781 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:36.993760    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:32:37.751620 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:37.751582    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" event={"ID":"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb","Type":"ContainerStarted","Data":"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"}
Sep 10 12:32:37.751792 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:37.751646    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" event={"ID":"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb","Type":"ContainerStarted","Data":"8832656c562e7014b271db90f5e6d9b0209e3c396633939f3fd2a07e1efb5374"}
Sep 10 12:32:37.751792 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:37.751673    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:32:37.781392 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:37.780471    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" podStartSLOduration=2.156122132 podStartE2EDuration="2.780451201s" totalImagesPullingTime="624.329069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:32:37.777509442 +0000 UTC m=+1385.558066937" watchObservedRunningTime="2026-09-10 12:32:37.780451201 +0000 UTC m=+1385.561008674"
Sep 10 12:32:43.761547 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:32:43.761519    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:33:05.067693 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.067659    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:33:05.071436 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.071416    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.074930 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.074904    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-da7b8f-serving-cert\""
Sep 10 12:33:05.090450 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.090425    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:33:05.158510 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.158478    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.158682 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.158547    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.158682 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.158621    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7c25\" (UniqueName: \"kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.158682 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.158664    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.259806 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.259762    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.260003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.259835    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7c25\" (UniqueName: \"kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.260003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.259875    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.260003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.259943    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.260671 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.260646    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.262323 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.262299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.262482 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.262464    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.268926 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.268902    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7c25\" (UniqueName: \"kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25\") pod \"maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.384402 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.384352    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:05.672967 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.672928    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:33:05.675735 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:33:05.675702    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b29c77_094c_418c_a059_b828b73c12ac.slice/crio-36a43294b3d133d5633ecaba6adcb66c263b95705316d8db19fed22e6c7023ca WatchSource:0}: Error finding container 36a43294b3d133d5633ecaba6adcb66c263b95705316d8db19fed22e6c7023ca: Status 404 returned error can't find the container with id 36a43294b3d133d5633ecaba6adcb66c263b95705316d8db19fed22e6c7023ca
Sep 10 12:33:05.916953 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:05.916920    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" event={"ID":"e6b29c77-094c-418c-a059-b828b73c12ac","Type":"ContainerStarted","Data":"36a43294b3d133d5633ecaba6adcb66c263b95705316d8db19fed22e6c7023ca"}
Sep 10 12:33:06.924495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:06.924408    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" event={"ID":"e6b29c77-094c-418c-a059-b828b73c12ac","Type":"ContainerStarted","Data":"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"}
Sep 10 12:33:06.924495 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:06.924485    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:06.948168 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:06.948119    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" podStartSLOduration=1.018894475 podStartE2EDuration="1.948105718s" totalImagesPullingTime="929.211243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:33:06.944999968 +0000 UTC m=+1414.725557453" watchObservedRunningTime="2026-09-10 12:33:06.948105718 +0000 UTC m=+1414.728663191"
Sep 10 12:33:12.935105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:12.935075    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:33:47.209296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.209190    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:33:47.210581 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.209794    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" podUID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" containerName="maas-api" containerID="cri-o://869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316" gracePeriod=30
Sep 10 12:33:47.485546 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.485518    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:33:47.579515 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.579479    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs\") pod \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") "
Sep 10 12:33:47.579726 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.579532    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld956\" (UniqueName: \"kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956\") pod \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") "
Sep 10 12:33:47.579726 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.579589    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") pod \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") "
Sep 10 12:33:47.579885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.579857    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca\") pod \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\" (UID: \"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb\") "
Sep 10 12:33:47.580283 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.580206    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" (UID: "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:33:47.580468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.580450    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:33:47.581813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.581779    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" (UID: "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:33:47.582112 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.582088    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" (UID: "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:33:47.582184 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.582136    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956" (OuterVolumeSpecName: "kube-api-access-ld956") pod "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" (UID: "35c4c1a4-0a23-41f2-afe1-8f9a613f77fb"). InnerVolumeSpecName "kube-api-access-ld956". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:33:47.681131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.681099    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:33:47.681131 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.681128    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld956\" (UniqueName: \"kubernetes.io/projected/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-kube-api-access-ld956\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:33:47.681322 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:47.681140    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:33:48.147555 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.147523    2551 generic.go:356] "Generic (PLEG): container finished" podID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" containerID="869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316" exitCode=0
Sep 10 12:33:48.147759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.147568    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"
Sep 10 12:33:48.147759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.147605    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" event={"ID":"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb","Type":"ContainerDied","Data":"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"}
Sep 10 12:33:48.147759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.147670    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c" event={"ID":"35c4c1a4-0a23-41f2-afe1-8f9a613f77fb","Type":"ContainerDied","Data":"8832656c562e7014b271db90f5e6d9b0209e3c396633939f3fd2a07e1efb5374"}
Sep 10 12:33:48.149014 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.147703    2551 scope.go:117] "RemoveContainer" containerID="869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"
Sep 10 12:33:48.158627 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.158610    2551 scope.go:117] "RemoveContainer" containerID="869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"
Sep 10 12:33:48.158925 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:33:48.158901    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316\": container with ID starting with 869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316 not found: ID does not exist" containerID="869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"
Sep 10 12:33:48.158974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.158934    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316"} err="failed to get container status \"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316\": rpc error: code = NotFound desc = could not find container \"869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316\": container with ID starting with 869920356d2eb1a54136dfae5369d53c95f8655cf7eea06c44e64ea9b546a316 not found: ID does not exist"
Sep 10 12:33:48.171749 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.171720    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:33:48.174337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.174315    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ff68c-6d8f4c4445-lln8c"]
Sep 10 12:33:48.724856 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:33:48.724820    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" path="/var/lib/kubelet/pods/35c4c1a4-0a23-41f2-afe1-8f9a613f77fb/volumes"
Sep 10 12:34:11.214791 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.214756    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:34:11.215226 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.215010    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" podUID="e6b29c77-094c-418c-a059-b828b73c12ac" containerName="maas-api" containerID="cri-o://84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a" gracePeriod=30
Sep 10 12:34:11.470578 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.470521    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:34:11.596837 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.596804    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca\") pod \"e6b29c77-094c-418c-a059-b828b73c12ac\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") "
Sep 10 12:34:11.597016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.596907    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs\") pod \"e6b29c77-094c-418c-a059-b828b73c12ac\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") "
Sep 10 12:34:11.597016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.596931    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls\") pod \"e6b29c77-094c-418c-a059-b828b73c12ac\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") "
Sep 10 12:34:11.597016 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.596956    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7c25\" (UniqueName: \"kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25\") pod \"e6b29c77-094c-418c-a059-b828b73c12ac\" (UID: \"e6b29c77-094c-418c-a059-b828b73c12ac\") "
Sep 10 12:34:11.597254 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.597229    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6b29c77-094c-418c-a059-b828b73c12ac" (UID: "e6b29c77-094c-418c-a059-b828b73c12ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:34:11.599114 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.599078    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e6b29c77-094c-418c-a059-b828b73c12ac" (UID: "e6b29c77-094c-418c-a059-b828b73c12ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:11.599267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.599162    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25" (OuterVolumeSpecName: "kube-api-access-b7c25") pod "e6b29c77-094c-418c-a059-b828b73c12ac" (UID: "e6b29c77-094c-418c-a059-b828b73c12ac"). InnerVolumeSpecName "kube-api-access-b7c25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:34:11.599267 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.599169    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6b29c77-094c-418c-a059-b828b73c12ac" (UID: "e6b29c77-094c-418c-a059-b828b73c12ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:11.698437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.698406    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:11.698437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.698433    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b29c77-094c-418c-a059-b828b73c12ac-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:11.698597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.698449    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7c25\" (UniqueName: \"kubernetes.io/projected/e6b29c77-094c-418c-a059-b828b73c12ac-kube-api-access-b7c25\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:11.698597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:11.698462    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b29c77-094c-418c-a059-b828b73c12ac-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:12.278494 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.278459    2551 generic.go:356] "Generic (PLEG): container finished" podID="e6b29c77-094c-418c-a059-b828b73c12ac" containerID="84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a" exitCode=0
Sep 10 12:34:12.278928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.278498    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"
Sep 10 12:34:12.278928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.278536    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" event={"ID":"e6b29c77-094c-418c-a059-b828b73c12ac","Type":"ContainerDied","Data":"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"}
Sep 10 12:34:12.278928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.278592    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz" event={"ID":"e6b29c77-094c-418c-a059-b828b73c12ac","Type":"ContainerDied","Data":"36a43294b3d133d5633ecaba6adcb66c263b95705316d8db19fed22e6c7023ca"}
Sep 10 12:34:12.278928 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.278615    2551 scope.go:117] "RemoveContainer" containerID="84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"
Sep 10 12:34:12.289122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.289106    2551 scope.go:117] "RemoveContainer" containerID="84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"
Sep 10 12:34:12.289418 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:34:12.289399    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a\": container with ID starting with 84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a not found: ID does not exist" containerID="84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"
Sep 10 12:34:12.289473 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.289425    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a"} err="failed to get container status \"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a\": rpc error: code = NotFound desc = could not find container \"84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a\": container with ID starting with 84b0bb4cf11ccad44bb2c62ea7574c784f1d9fce1be34eba961ccd75e305d43a not found: ID does not exist"
Sep 10 12:34:12.304410 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.304391    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:34:12.314084 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.314059    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da7b8f-8654494c4-7mlgz"]
Sep 10 12:34:12.724250 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:12.724216    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b29c77-094c-418c-a059-b828b73c12ac" path="/var/lib/kubelet/pods/e6b29c77-094c-418c-a059-b828b73c12ac/volumes"
Sep 10 12:34:30.455483 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.455447    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:34:30.456088 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.455746    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" podUID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" containerName="maas-api" containerID="cri-o://9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78" gracePeriod=30
Sep 10 12:34:30.721110 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.721049    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:34:30.885363 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.885331    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs\") pod \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") "
Sep 10 12:34:30.885567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.885419    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") pod \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") "
Sep 10 12:34:30.885567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.885508    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgmnh\" (UniqueName: \"kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh\") pod \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") "
Sep 10 12:34:30.885567 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.885537    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca\") pod \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\" (UID: \"7c6c2880-9b37-4467-b06e-82ad1e9a601c\") "
Sep 10 12:34:30.885999 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.885964    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c6c2880-9b37-4467-b06e-82ad1e9a601c" (UID: "7c6c2880-9b37-4467-b06e-82ad1e9a601c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:34:30.886428 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.886321    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c6c2880-9b37-4467-b06e-82ad1e9a601c-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:30.887737 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.887715    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c6c2880-9b37-4467-b06e-82ad1e9a601c" (UID: "7c6c2880-9b37-4467-b06e-82ad1e9a601c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:30.887827 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.887812    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c6c2880-9b37-4467-b06e-82ad1e9a601c" (UID: "7c6c2880-9b37-4467-b06e-82ad1e9a601c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:30.887984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.887958    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh" (OuterVolumeSpecName: "kube-api-access-tgmnh") pod "7c6c2880-9b37-4467-b06e-82ad1e9a601c" (UID: "7c6c2880-9b37-4467-b06e-82ad1e9a601c"). InnerVolumeSpecName "kube-api-access-tgmnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:34:30.987711 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.987624    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgmnh\" (UniqueName: \"kubernetes.io/projected/7c6c2880-9b37-4467-b06e-82ad1e9a601c-kube-api-access-tgmnh\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:30.987711 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.987655    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:30.987711 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:30.987665    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c6c2880-9b37-4467-b06e-82ad1e9a601c-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:31.384418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.384357    2551 generic.go:356] "Generic (PLEG): container finished" podID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" containerID="9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78" exitCode=0
Sep 10 12:34:31.384597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.384435    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"
Sep 10 12:34:31.384597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.384443    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" event={"ID":"7c6c2880-9b37-4467-b06e-82ad1e9a601c","Type":"ContainerDied","Data":"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"}
Sep 10 12:34:31.384597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.384524    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g" event={"ID":"7c6c2880-9b37-4467-b06e-82ad1e9a601c","Type":"ContainerDied","Data":"772fb93b39cd0dd686aff59757cc7e19ceb84bff3704d88bacd1d905de734d0b"}
Sep 10 12:34:31.384597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.384541    2551 scope.go:117] "RemoveContainer" containerID="9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"
Sep 10 12:34:31.394252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.394233    2551 scope.go:117] "RemoveContainer" containerID="9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"
Sep 10 12:34:31.394596 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:34:31.394574    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78\": container with ID starting with 9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78 not found: ID does not exist" containerID="9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"
Sep 10 12:34:31.394675 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.394608    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78"} err="failed to get container status \"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78\": rpc error: code = NotFound desc = could not find container \"9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78\": container with ID starting with 9ef3c6aad00e989120e8cd07432a9ce4b1cca681e0d59f3dfd3a373a2a2b4f78 not found: ID does not exist"
Sep 10 12:34:31.409214 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.409177    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:34:31.413818 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:31.413781    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d1e553-75dd474df-fxc7g"]
Sep 10 12:34:32.723665 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:32.723622    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" path="/var/lib/kubelet/pods/7c6c2880-9b37-4467-b06e-82ad1e9a601c/volumes"
Sep 10 12:34:32.759255 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:32.759225    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:34:32.763461 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:32.763432    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:34:32.763936 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:32.763918    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:34:32.767641 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:32.767620    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:34:44.455200 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.455160    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:34:44.458500 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458475    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" containerName="maas-api"
Sep 10 12:34:44.458500 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458499    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" containerName="maas-api"
Sep 10 12:34:44.458657 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458517    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b29c77-094c-418c-a059-b828b73c12ac" containerName="maas-api"
Sep 10 12:34:44.458657 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458526    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b29c77-094c-418c-a059-b828b73c12ac" containerName="maas-api"
Sep 10 12:34:44.458657 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458548    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" containerName="maas-api"
Sep 10 12:34:44.458657 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458557    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" containerName="maas-api"
Sep 10 12:34:44.458807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458668    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6c2880-9b37-4467-b06e-82ad1e9a601c" containerName="maas-api"
Sep 10 12:34:44.458807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458683    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b29c77-094c-418c-a059-b828b73c12ac" containerName="maas-api"
Sep 10 12:34:44.458807 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.458694    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c4c1a4-0a23-41f2-afe1-8f9a613f77fb" containerName="maas-api"
Sep 10 12:34:44.462528 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.462504    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.465666 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.465638    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-3db7ea-serving-cert\""
Sep 10 12:34:44.472682 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.472658    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:34:44.507691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.507657    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z59qz\" (UniqueName: \"kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.507847 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.507720    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.507847 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.507787    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.507961 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.507938    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.608776 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.608740    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z59qz\" (UniqueName: \"kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.608976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.608806    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.608976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.608858    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.608976 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.608929    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.609144 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:34:44.608979    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-serving-cert: secret "maas-api-e2e-ipp-a-3db7ea-serving-cert" not found
Sep 10 12:34:44.609144 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:34:44.609058    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls podName:62da0cf6-bf80-4bb5-92e8-b9b80190e94a nodeName:}" failed. No retries permitted until 2026-09-10 12:34:45.109038657 +0000 UTC m=+1512.889596114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls") pod "maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" (UID: "62da0cf6-bf80-4bb5-92e8-b9b80190e94a") : secret "maas-api-e2e-ipp-a-3db7ea-serving-cert" not found
Sep 10 12:34:44.609629 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.609608    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.611217 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.611196    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:44.631633 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:44.631609    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z59qz\" (UniqueName: \"kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:45.114497 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:45.114457    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:45.116758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:45.116739    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:45.375311 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:45.375206    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:45.661855 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:45.661811    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:34:45.663438 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:34:45.663403    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62da0cf6_bf80_4bb5_92e8_b9b80190e94a.slice/crio-c5cf8957c55d21dc62c2d495a146f15be5783c6466025255c49ae2244672fa11 WatchSource:0}: Error finding container c5cf8957c55d21dc62c2d495a146f15be5783c6466025255c49ae2244672fa11: Status 404 returned error can't find the container with id c5cf8957c55d21dc62c2d495a146f15be5783c6466025255c49ae2244672fa11
Sep 10 12:34:46.470917 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:46.470887    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" event={"ID":"62da0cf6-bf80-4bb5-92e8-b9b80190e94a","Type":"ContainerStarted","Data":"c5cf8957c55d21dc62c2d495a146f15be5783c6466025255c49ae2244672fa11"}
Sep 10 12:34:47.477318 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:47.477281    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" event={"ID":"62da0cf6-bf80-4bb5-92e8-b9b80190e94a","Type":"ContainerStarted","Data":"75e88ab13af4df014acdf58130cdefda341c347135e999e76916d731662b021f"}
Sep 10 12:34:47.477734 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:47.477343    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:34:47.497522 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:47.497469    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" podStartSLOduration=2.7314455349999998 podStartE2EDuration="3.49745119s" totalImagesPullingTime="766.005655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:34:47.496485875 +0000 UTC m=+1515.277043374" watchObservedRunningTime="2026-09-10 12:34:47.49745119 +0000 UTC m=+1515.278008714"
Sep 10 12:34:49.794040 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:49.794010    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:34:49.794518 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:49.794192    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" podUID="43abb84a-421b-486b-b1d6-9165b361ee72" containerName="maas-api" containerID="cri-o://6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299" gracePeriod=30
Sep 10 12:34:50.044858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.044805    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:34:50.159780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.159751    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca\") pod \"43abb84a-421b-486b-b1d6-9165b361ee72\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") "
Sep 10 12:34:50.159780 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.159783    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") pod \"43abb84a-421b-486b-b1d6-9165b361ee72\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") "
Sep 10 12:34:50.159985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.159919    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwngs\" (UniqueName: \"kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs\") pod \"43abb84a-421b-486b-b1d6-9165b361ee72\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") "
Sep 10 12:34:50.159985 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.159959    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs\") pod \"43abb84a-421b-486b-b1d6-9165b361ee72\" (UID: \"43abb84a-421b-486b-b1d6-9165b361ee72\") "
Sep 10 12:34:50.160174 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.160151    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43abb84a-421b-486b-b1d6-9165b361ee72" (UID: "43abb84a-421b-486b-b1d6-9165b361ee72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:34:50.161997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.161965    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43abb84a-421b-486b-b1d6-9165b361ee72" (UID: "43abb84a-421b-486b-b1d6-9165b361ee72"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:50.161997 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.161979    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43abb84a-421b-486b-b1d6-9165b361ee72" (UID: "43abb84a-421b-486b-b1d6-9165b361ee72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:34:50.162149 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.162030    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs" (OuterVolumeSpecName: "kube-api-access-wwngs") pod "43abb84a-421b-486b-b1d6-9165b361ee72" (UID: "43abb84a-421b-486b-b1d6-9165b361ee72"). InnerVolumeSpecName "kube-api-access-wwngs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:34:50.260598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.260569    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwngs\" (UniqueName: \"kubernetes.io/projected/43abb84a-421b-486b-b1d6-9165b361ee72-kube-api-access-wwngs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:50.260598 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.260593    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:50.260759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.260605    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43abb84a-421b-486b-b1d6-9165b361ee72-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:50.260759 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.260613    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43abb84a-421b-486b-b1d6-9165b361ee72-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:34:50.495481 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.495450    2551 generic.go:356] "Generic (PLEG): container finished" podID="43abb84a-421b-486b-b1d6-9165b361ee72" containerID="6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299" exitCode=0
Sep 10 12:34:50.495616 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.495501    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"
Sep 10 12:34:50.495616 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.495543    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" event={"ID":"43abb84a-421b-486b-b1d6-9165b361ee72","Type":"ContainerDied","Data":"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"}
Sep 10 12:34:50.495616 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.495601    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m" event={"ID":"43abb84a-421b-486b-b1d6-9165b361ee72","Type":"ContainerDied","Data":"98a6197f5fc51852bdd47f035254d20f2f29875ba363509677a1f790c0f084de"}
Sep 10 12:34:50.495723 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.495628    2551 scope.go:117] "RemoveContainer" containerID="6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"
Sep 10 12:34:50.505466 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.505449    2551 scope.go:117] "RemoveContainer" containerID="6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"
Sep 10 12:34:50.505729 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:34:50.505707    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299\": container with ID starting with 6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299 not found: ID does not exist" containerID="6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"
Sep 10 12:34:50.505819 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.505736    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299"} err="failed to get container status \"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299\": rpc error: code = NotFound desc = could not find container \"6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299\": container with ID starting with 6eb594f141b7dce2c317d17de7b636fbc837f766559cfc8a99225e7a80edb299 not found: ID does not exist"
Sep 10 12:34:50.539236 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.539213    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:34:50.543597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.543578    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a08ea2-5f4f7855cd-77v5m"]
Sep 10 12:34:50.723156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:50.723132    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43abb84a-421b-486b-b1d6-9165b361ee72" path="/var/lib/kubelet/pods/43abb84a-421b-486b-b1d6-9165b361ee72/volumes"
Sep 10 12:34:53.489484 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:34:53.489450    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:35:13.936714 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.936678    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:35:13.937132 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.937119    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43abb84a-421b-486b-b1d6-9165b361ee72" containerName="maas-api"
Sep 10 12:35:13.937175 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.937134    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43abb84a-421b-486b-b1d6-9165b361ee72" containerName="maas-api"
Sep 10 12:35:13.937229 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.937219    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="43abb84a-421b-486b-b1d6-9165b361ee72" containerName="maas-api"
Sep 10 12:35:13.940572 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.940554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:13.945232 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.945215    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a5be11-serving-cert\""
Sep 10 12:35:13.963786 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:13.963752    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:35:14.068007 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.067975    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.068208 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.068017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.068208 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.068053    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.068208 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.068102    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjhb8\" (UniqueName: \"kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.169477 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.169439    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.169673 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.169486    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.169673 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.169519    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjhb8\" (UniqueName: \"kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.169673 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.169660    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.170325 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.170299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.172066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.172038    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.172066 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.172042    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.180858 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.180834    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjhb8\" (UniqueName: \"kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8\") pod \"maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.250455 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.250361    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:14.543676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.543634    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:35:14.544111 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:35:14.544057    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fff67c_b2b6_4f05_91a0_84009072c16a.slice/crio-6e48e6db8f2375a5e5c971c93a2e0742de85ab5ddca97f4bae8f5a713f6a62d7 WatchSource:0}: Error finding container 6e48e6db8f2375a5e5c971c93a2e0742de85ab5ddca97f4bae8f5a713f6a62d7: Status 404 returned error can't find the container with id 6e48e6db8f2375a5e5c971c93a2e0742de85ab5ddca97f4bae8f5a713f6a62d7
Sep 10 12:35:14.621611 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:14.621583    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" event={"ID":"36fff67c-b2b6-4f05-91a0-84009072c16a","Type":"ContainerStarted","Data":"6e48e6db8f2375a5e5c971c93a2e0742de85ab5ddca97f4bae8f5a713f6a62d7"}
Sep 10 12:35:15.628922 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:15.628882    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" event={"ID":"36fff67c-b2b6-4f05-91a0-84009072c16a","Type":"ContainerStarted","Data":"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"}
Sep 10 12:35:15.629390 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:15.629016    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:15.657538 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:15.657490    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" podStartSLOduration=2.072796417 podStartE2EDuration="2.657474934s" totalImagesPullingTime="584.678517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:35:15.657347105 +0000 UTC m=+1543.437904578" watchObservedRunningTime="2026-09-10 12:35:15.657474934 +0000 UTC m=+1543.438032406"
Sep 10 12:35:21.639336 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:21.639310    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:35:42.465166 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:35:42.465133    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:36:18.348884 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:18.348851    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:36:23.414799 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.414753    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:23.419096 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.419070    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.422086 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.422057    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-47deee-serving-cert\""
Sep 10 12:36:23.485476 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.485437    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:23.490439 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.490407    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.490617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.490488    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.490617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.490526    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlrfg\" (UniqueName: \"kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.490617 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.490603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.591445 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.591335    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.591445 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.591443    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.591739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.591469    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlrfg\" (UniqueName: \"kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.591739 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.591528    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.591739 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:36:23.591705    2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-serving-cert: secret "maas-api-e2e-ipp-cleanup-47deee-serving-cert" not found
Sep 10 12:36:23.591890 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:36:23.591788    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls podName:c061c338-e88c-40b5-8bf7-1e238151238b nodeName:}" failed. No retries permitted until 2026-09-10 12:36:24.091768102 +0000 UTC m=+1611.872325556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" (UID: "c061c338-e88c-40b5-8bf7-1e238151238b") : secret "maas-api-e2e-ipp-cleanup-47deee-serving-cert" not found
Sep 10 12:36:23.592138 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.592111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.594038 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.594012    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:23.624878 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:23.624842    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlrfg\" (UniqueName: \"kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:24.096969 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:24.096929    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:24.099480 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:24.099456    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:24.331366 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:24.331325    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:24.632743 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:24.632715    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:24.634630 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:36:24.634568    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc061c338_e88c_40b5_8bf7_1e238151238b.slice/crio-d4a96a4c0fb6df93431702a47ba49f6af57bfe7eeeda1c6495160977292f809d WatchSource:0}: Error finding container d4a96a4c0fb6df93431702a47ba49f6af57bfe7eeeda1c6495160977292f809d: Status 404 returned error can't find the container with id d4a96a4c0fb6df93431702a47ba49f6af57bfe7eeeda1c6495160977292f809d
Sep 10 12:36:25.008023 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:25.007908    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" event={"ID":"c061c338-e88c-40b5-8bf7-1e238151238b","Type":"ContainerStarted","Data":"d4a96a4c0fb6df93431702a47ba49f6af57bfe7eeeda1c6495160977292f809d"}
Sep 10 12:36:26.015309 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:26.015271    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" event={"ID":"c061c338-e88c-40b5-8bf7-1e238151238b","Type":"ContainerStarted","Data":"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"}
Sep 10 12:36:26.015719 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:26.015332    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:26.058905 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:26.058853    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" podStartSLOduration=2.437834293 podStartE2EDuration="3.058839001s" totalImagesPullingTime="621.004708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:36:26.057586851 +0000 UTC m=+1613.838144354" watchObservedRunningTime="2026-09-10 12:36:26.058839001 +0000 UTC m=+1613.839396478"
Sep 10 12:36:32.026532 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:32.026500    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:54.105241 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.105206    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:54.105859 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.105464    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" podUID="c061c338-e88c-40b5-8bf7-1e238151238b" containerName="maas-api" containerID="cri-o://4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33" gracePeriod=30
Sep 10 12:36:54.365909 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.365854    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:54.471531 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.471499    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlrfg\" (UniqueName: \"kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg\") pod \"c061c338-e88c-40b5-8bf7-1e238151238b\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") "
Sep 10 12:36:54.471696 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.471577    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") pod \"c061c338-e88c-40b5-8bf7-1e238151238b\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") "
Sep 10 12:36:54.471696 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.471639    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs\") pod \"c061c338-e88c-40b5-8bf7-1e238151238b\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") "
Sep 10 12:36:54.471696 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.471656    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca\") pod \"c061c338-e88c-40b5-8bf7-1e238151238b\" (UID: \"c061c338-e88c-40b5-8bf7-1e238151238b\") "
Sep 10 12:36:54.472042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.472011    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c061c338-e88c-40b5-8bf7-1e238151238b" (UID: "c061c338-e88c-40b5-8bf7-1e238151238b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:36:54.473704 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.473676    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg" (OuterVolumeSpecName: "kube-api-access-xlrfg") pod "c061c338-e88c-40b5-8bf7-1e238151238b" (UID: "c061c338-e88c-40b5-8bf7-1e238151238b"). InnerVolumeSpecName "kube-api-access-xlrfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:36:54.473801 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.473714    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c061c338-e88c-40b5-8bf7-1e238151238b" (UID: "c061c338-e88c-40b5-8bf7-1e238151238b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:36:54.473842 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.473810    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c061c338-e88c-40b5-8bf7-1e238151238b" (UID: "c061c338-e88c-40b5-8bf7-1e238151238b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:36:54.572850 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.572817    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:36:54.572850 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.572844    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c061c338-e88c-40b5-8bf7-1e238151238b-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:36:54.573001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.572855    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061c338-e88c-40b5-8bf7-1e238151238b-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:36:54.573001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:54.572865    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlrfg\" (UniqueName: \"kubernetes.io/projected/c061c338-e88c-40b5-8bf7-1e238151238b-kube-api-access-xlrfg\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:36:55.176458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.176427    2551 generic.go:356] "Generic (PLEG): container finished" podID="c061c338-e88c-40b5-8bf7-1e238151238b" containerID="4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33" exitCode=0
Sep 10 12:36:55.176839 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.176471    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"
Sep 10 12:36:55.176839 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.176521    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" event={"ID":"c061c338-e88c-40b5-8bf7-1e238151238b","Type":"ContainerDied","Data":"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"}
Sep 10 12:36:55.176839 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.176568    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k" event={"ID":"c061c338-e88c-40b5-8bf7-1e238151238b","Type":"ContainerDied","Data":"d4a96a4c0fb6df93431702a47ba49f6af57bfe7eeeda1c6495160977292f809d"}
Sep 10 12:36:55.176839 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.176591    2551 scope.go:117] "RemoveContainer" containerID="4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"
Sep 10 12:36:55.185874 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.185856    2551 scope.go:117] "RemoveContainer" containerID="4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"
Sep 10 12:36:55.186141 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:36:55.186124    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33\": container with ID starting with 4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33 not found: ID does not exist" containerID="4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"
Sep 10 12:36:55.186185 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.186148    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33"} err="failed to get container status \"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33\": rpc error: code = NotFound desc = could not find container \"4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33\": container with ID starting with 4944604e99fa8fdb41ba64894cec1aa062e4c48d8388b42d6a6a50905be7fa33 not found: ID does not exist"
Sep 10 12:36:55.199757 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.199736    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:55.201656 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:55.201639    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-47deee-57cdbcfb55-gh92k"]
Sep 10 12:36:56.723843 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:36:56.723810    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c061c338-e88c-40b5-8bf7-1e238151238b" path="/var/lib/kubelet/pods/c061c338-e88c-40b5-8bf7-1e238151238b/volumes"
Sep 10 12:37:21.175596 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.175564    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:37:21.176565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.175781    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" podUID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" containerName="maas-api" containerID="cri-o://75e88ab13af4df014acdf58130cdefda341c347135e999e76916d731662b021f" gracePeriod=30
Sep 10 12:37:21.317836 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.317800    2551 generic.go:356] "Generic (PLEG): container finished" podID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" containerID="75e88ab13af4df014acdf58130cdefda341c347135e999e76916d731662b021f" exitCode=0
Sep 10 12:37:21.318012 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.317988    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" event={"ID":"62da0cf6-bf80-4bb5-92e8-b9b80190e94a","Type":"ContainerDied","Data":"75e88ab13af4df014acdf58130cdefda341c347135e999e76916d731662b021f"}
Sep 10 12:37:21.433716 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.433663    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:37:21.517911 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.517886    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z59qz\" (UniqueName: \"kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz\") pod \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") "
Sep 10 12:37:21.518067 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.517930    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs\") pod \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") "
Sep 10 12:37:21.518067 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.517952    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca\") pod \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") "
Sep 10 12:37:21.518155 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.518083    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") pod \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\" (UID: \"62da0cf6-bf80-4bb5-92e8-b9b80190e94a\") "
Sep 10 12:37:21.518332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.518304    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62da0cf6-bf80-4bb5-92e8-b9b80190e94a" (UID: "62da0cf6-bf80-4bb5-92e8-b9b80190e94a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:37:21.518510 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.518413    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:37:21.520050 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.520027    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "62da0cf6-bf80-4bb5-92e8-b9b80190e94a" (UID: "62da0cf6-bf80-4bb5-92e8-b9b80190e94a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:37:21.520163 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.520058    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62da0cf6-bf80-4bb5-92e8-b9b80190e94a" (UID: "62da0cf6-bf80-4bb5-92e8-b9b80190e94a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:37:21.520163 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.520075    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz" (OuterVolumeSpecName: "kube-api-access-z59qz") pod "62da0cf6-bf80-4bb5-92e8-b9b80190e94a" (UID: "62da0cf6-bf80-4bb5-92e8-b9b80190e94a"). InnerVolumeSpecName "kube-api-access-z59qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:37:21.619230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.619200    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:37:21.619230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.619229    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z59qz\" (UniqueName: \"kubernetes.io/projected/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-kube-api-access-z59qz\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:37:21.619431 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:21.619240    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62da0cf6-bf80-4bb5-92e8-b9b80190e94a-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:37:22.324679 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.324644    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"
Sep 10 12:37:22.325081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.324679    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g" event={"ID":"62da0cf6-bf80-4bb5-92e8-b9b80190e94a","Type":"ContainerDied","Data":"c5cf8957c55d21dc62c2d495a146f15be5783c6466025255c49ae2244672fa11"}
Sep 10 12:37:22.325081 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.324741    2551 scope.go:117] "RemoveContainer" containerID="75e88ab13af4df014acdf58130cdefda341c347135e999e76916d731662b021f"
Sep 10 12:37:22.363918 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.363889    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:37:22.369552 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.369532    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3db7ea-7b94949566-7vg2g"]
Sep 10 12:37:22.723937 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:37:22.723909    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" path="/var/lib/kubelet/pods/62da0cf6-bf80-4bb5-92e8-b9b80190e94a/volumes"
Sep 10 12:38:21.808440 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:21.808403    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:38:21.809025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:21.808598    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" podUID="36fff67c-b2b6-4f05-91a0-84009072c16a" containerName="maas-api" containerID="cri-o://ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110" gracePeriod=30
Sep 10 12:38:22.059175 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.059121    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:38:22.125722 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.125687    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjhb8\" (UniqueName: \"kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8\") pod \"36fff67c-b2b6-4f05-91a0-84009072c16a\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") "
Sep 10 12:38:22.125875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.125746    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs\") pod \"36fff67c-b2b6-4f05-91a0-84009072c16a\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") "
Sep 10 12:38:22.125875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.125814    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca\") pod \"36fff67c-b2b6-4f05-91a0-84009072c16a\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") "
Sep 10 12:38:22.125875 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.125838    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls\") pod \"36fff67c-b2b6-4f05-91a0-84009072c16a\" (UID: \"36fff67c-b2b6-4f05-91a0-84009072c16a\") "
Sep 10 12:38:22.126179 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.126150    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36fff67c-b2b6-4f05-91a0-84009072c16a" (UID: "36fff67c-b2b6-4f05-91a0-84009072c16a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:38:22.127894 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.127869    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36fff67c-b2b6-4f05-91a0-84009072c16a" (UID: "36fff67c-b2b6-4f05-91a0-84009072c16a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:22.128022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.127970    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8" (OuterVolumeSpecName: "kube-api-access-pjhb8") pod "36fff67c-b2b6-4f05-91a0-84009072c16a" (UID: "36fff67c-b2b6-4f05-91a0-84009072c16a"). InnerVolumeSpecName "kube-api-access-pjhb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:38:22.128022 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.127971    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "36fff67c-b2b6-4f05-91a0-84009072c16a" (UID: "36fff67c-b2b6-4f05-91a0-84009072c16a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:22.226943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.226910    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjhb8\" (UniqueName: \"kubernetes.io/projected/36fff67c-b2b6-4f05-91a0-84009072c16a-kube-api-access-pjhb8\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:22.226943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.226939    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:22.226943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.226947    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fff67c-b2b6-4f05-91a0-84009072c16a-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:22.227156 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.226955    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fff67c-b2b6-4f05-91a0-84009072c16a-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:22.641952 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.641915    2551 generic.go:356] "Generic (PLEG): container finished" podID="36fff67c-b2b6-4f05-91a0-84009072c16a" containerID="ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110" exitCode=0
Sep 10 12:38:22.642122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.641963    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"
Sep 10 12:38:22.642122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.641986    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" event={"ID":"36fff67c-b2b6-4f05-91a0-84009072c16a","Type":"ContainerDied","Data":"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"}
Sep 10 12:38:22.642122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.642037    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf" event={"ID":"36fff67c-b2b6-4f05-91a0-84009072c16a","Type":"ContainerDied","Data":"6e48e6db8f2375a5e5c971c93a2e0742de85ab5ddca97f4bae8f5a713f6a62d7"}
Sep 10 12:38:22.642122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.642066    2551 scope.go:117] "RemoveContainer" containerID="ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"
Sep 10 12:38:22.651670 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.651654    2551 scope.go:117] "RemoveContainer" containerID="ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"
Sep 10 12:38:22.651937 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:38:22.651917    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110\": container with ID starting with ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110 not found: ID does not exist" containerID="ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"
Sep 10 12:38:22.651980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.651946    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110"} err="failed to get container status \"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110\": rpc error: code = NotFound desc = could not find container \"ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110\": container with ID starting with ad1b99f4ffe7d47ec3a639cb41dc3e2c7f1a82b25727868519e10fa0af31e110 not found: ID does not exist"
Sep 10 12:38:22.664896 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.664872    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:38:22.669420 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.669389    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a5be11-75b4747c86-mtvvf"]
Sep 10 12:38:22.723500 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:22.723469    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36fff67c-b2b6-4f05-91a0-84009072c16a" path="/var/lib/kubelet/pods/36fff67c-b2b6-4f05-91a0-84009072c16a/volumes"
Sep 10 12:38:39.688265 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:39.688224    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:38:39.688766 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:39.688494    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" podUID="c354b2d2-723a-41ba-9322-6fea1210cfc4" containerName="maas-api" containerID="cri-o://869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e" gracePeriod=30
Sep 10 12:38:39.944829 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:39.944777    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:38:40.070447 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.070420    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2wdt\" (UniqueName: \"kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt\") pod \"c354b2d2-723a-41ba-9322-6fea1210cfc4\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") "
Sep 10 12:38:40.070589 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.070495    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs\") pod \"c354b2d2-723a-41ba-9322-6fea1210cfc4\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") "
Sep 10 12:38:40.070589 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.070533    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") pod \"c354b2d2-723a-41ba-9322-6fea1210cfc4\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") "
Sep 10 12:38:40.070589 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.070553    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca\") pod \"c354b2d2-723a-41ba-9322-6fea1210cfc4\" (UID: \"c354b2d2-723a-41ba-9322-6fea1210cfc4\") "
Sep 10 12:38:40.070943 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.070919    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c354b2d2-723a-41ba-9322-6fea1210cfc4" (UID: "c354b2d2-723a-41ba-9322-6fea1210cfc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:38:40.072534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.072508    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c354b2d2-723a-41ba-9322-6fea1210cfc4" (UID: "c354b2d2-723a-41ba-9322-6fea1210cfc4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:40.072534 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.072515    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt" (OuterVolumeSpecName: "kube-api-access-n2wdt") pod "c354b2d2-723a-41ba-9322-6fea1210cfc4" (UID: "c354b2d2-723a-41ba-9322-6fea1210cfc4"). InnerVolumeSpecName "kube-api-access-n2wdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:38:40.072655 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.072559    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c354b2d2-723a-41ba-9322-6fea1210cfc4" (UID: "c354b2d2-723a-41ba-9322-6fea1210cfc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:40.172064 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.172039    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2wdt\" (UniqueName: \"kubernetes.io/projected/c354b2d2-723a-41ba-9322-6fea1210cfc4-kube-api-access-n2wdt\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:40.172064 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.172062    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:40.172192 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.172072    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c354b2d2-723a-41ba-9322-6fea1210cfc4-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:40.172192 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.172080    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c354b2d2-723a-41ba-9322-6fea1210cfc4-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:40.735031 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.735001    2551 generic.go:356] "Generic (PLEG): container finished" podID="c354b2d2-723a-41ba-9322-6fea1210cfc4" containerID="869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e" exitCode=0
Sep 10 12:38:40.735410 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.735044    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"
Sep 10 12:38:40.735410 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.735071    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" event={"ID":"c354b2d2-723a-41ba-9322-6fea1210cfc4","Type":"ContainerDied","Data":"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"}
Sep 10 12:38:40.735410 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.735109    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk" event={"ID":"c354b2d2-723a-41ba-9322-6fea1210cfc4","Type":"ContainerDied","Data":"a5c14c488dca69e6ed1c2bc0c2c19beef36a7cc418f4b0d0f2361b394abea2a7"}
Sep 10 12:38:40.735410 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.735130    2551 scope.go:117] "RemoveContainer" containerID="869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"
Sep 10 12:38:40.743838 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.743820    2551 scope.go:117] "RemoveContainer" containerID="869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"
Sep 10 12:38:40.744098 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:38:40.744077    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e\": container with ID starting with 869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e not found: ID does not exist" containerID="869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"
Sep 10 12:38:40.744198 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.744104    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e"} err="failed to get container status \"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e\": rpc error: code = NotFound desc = could not find container \"869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e\": container with ID starting with 869bd495b7be806a4c3307caadfcef5db2004da3164c255e8e3b01022787002e not found: ID does not exist"
Sep 10 12:38:40.753203 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.753183    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:38:40.757718 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:40.757694    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-18c2d2-cb99f9f6c-bn6lk"]
Sep 10 12:38:42.723583 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:42.723548    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c354b2d2-723a-41ba-9322-6fea1210cfc4" path="/var/lib/kubelet/pods/c354b2d2-723a-41ba-9322-6fea1210cfc4/volumes"
Sep 10 12:38:57.787651 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:57.787617    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:38:57.788127 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:57.787869    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" podUID="a2b1b76a-cf06-4771-9588-fa33eef78a54" containerName="maas-api" containerID="cri-o://195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7" gracePeriod=30
Sep 10 12:38:58.043116 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.043050    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:38:58.110914 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.110891    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs\") pod \"a2b1b76a-cf06-4771-9588-fa33eef78a54\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") "
Sep 10 12:38:58.111042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.110935    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sxp6\" (UniqueName: \"kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6\") pod \"a2b1b76a-cf06-4771-9588-fa33eef78a54\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") "
Sep 10 12:38:58.111042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.110960    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") pod \"a2b1b76a-cf06-4771-9588-fa33eef78a54\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") "
Sep 10 12:38:58.111042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.110988    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca\") pod \"a2b1b76a-cf06-4771-9588-fa33eef78a54\" (UID: \"a2b1b76a-cf06-4771-9588-fa33eef78a54\") "
Sep 10 12:38:58.111402 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.111358    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2b1b76a-cf06-4771-9588-fa33eef78a54" (UID: "a2b1b76a-cf06-4771-9588-fa33eef78a54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:38:58.112927 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.112900    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2b1b76a-cf06-4771-9588-fa33eef78a54" (UID: "a2b1b76a-cf06-4771-9588-fa33eef78a54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:58.113029 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.112924    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2b1b76a-cf06-4771-9588-fa33eef78a54" (UID: "a2b1b76a-cf06-4771-9588-fa33eef78a54"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:38:58.113122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.113103    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6" (OuterVolumeSpecName: "kube-api-access-6sxp6") pod "a2b1b76a-cf06-4771-9588-fa33eef78a54" (UID: "a2b1b76a-cf06-4771-9588-fa33eef78a54"). InnerVolumeSpecName "kube-api-access-6sxp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:38:58.212457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.212435    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sxp6\" (UniqueName: \"kubernetes.io/projected/a2b1b76a-cf06-4771-9588-fa33eef78a54-kube-api-access-6sxp6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:58.212457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.212459    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:58.212600 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.212467    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2b1b76a-cf06-4771-9588-fa33eef78a54-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:58.212600 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.212475    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2b1b76a-cf06-4771-9588-fa33eef78a54-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:38:58.821451 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.821417    2551 generic.go:356] "Generic (PLEG): container finished" podID="a2b1b76a-cf06-4771-9588-fa33eef78a54" containerID="195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7" exitCode=0
Sep 10 12:38:58.821817 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.821460    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"
Sep 10 12:38:58.821817 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.821493    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" event={"ID":"a2b1b76a-cf06-4771-9588-fa33eef78a54","Type":"ContainerDied","Data":"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"}
Sep 10 12:38:58.821817 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.821541    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf" event={"ID":"a2b1b76a-cf06-4771-9588-fa33eef78a54","Type":"ContainerDied","Data":"08713d9f36f822bdb4029f91a55b065691407db3c8e8b757864d1da5e24f501b"}
Sep 10 12:38:58.821817 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.821563    2551 scope.go:117] "RemoveContainer" containerID="195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"
Sep 10 12:38:58.830593 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.830575    2551 scope.go:117] "RemoveContainer" containerID="195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"
Sep 10 12:38:58.830901 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:38:58.830879    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7\": container with ID starting with 195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7 not found: ID does not exist" containerID="195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"
Sep 10 12:38:58.830958 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.830909    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7"} err="failed to get container status \"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7\": rpc error: code = NotFound desc = could not find container \"195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7\": container with ID starting with 195b41b9336522362c1c085fe3e209ffaf86aad7df576b5cb38c4df0c5dc50f7 not found: ID does not exist"
Sep 10 12:38:58.839821 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.839800    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:38:58.843987 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:38:58.843963    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f89a79-79df5688f8-b4lmf"]
Sep 10 12:39:00.723458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:00.723424    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2b1b76a-cf06-4771-9588-fa33eef78a54" path="/var/lib/kubelet/pods/a2b1b76a-cf06-4771-9588-fa33eef78a54/volumes"
Sep 10 12:39:07.477075 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.477035    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:39:07.477553 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.477280    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" podUID="9faa6201-2130-402d-9408-acb9b353f5ba" containerName="manager" containerID="cri-o://5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8" gracePeriod=30
Sep 10 12:39:07.728676 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.728622    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:39:07.788418 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.788385    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs\") pod \"9faa6201-2130-402d-9408-acb9b353f5ba\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") "
Sep 10 12:39:07.788571 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.788481    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xgl9\" (UniqueName: \"kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9\") pod \"9faa6201-2130-402d-9408-acb9b353f5ba\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") "
Sep 10 12:39:07.788571 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.788523    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert\") pod \"9faa6201-2130-402d-9408-acb9b353f5ba\" (UID: \"9faa6201-2130-402d-9408-acb9b353f5ba\") "
Sep 10 12:39:07.798990 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.798928    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9faa6201-2130-402d-9408-acb9b353f5ba" (UID: "9faa6201-2130-402d-9408-acb9b353f5ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:39:07.799358 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.799330    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9faa6201-2130-402d-9408-acb9b353f5ba" (UID: "9faa6201-2130-402d-9408-acb9b353f5ba"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:39:07.799468 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.799396    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9" (OuterVolumeSpecName: "kube-api-access-8xgl9") pod "9faa6201-2130-402d-9408-acb9b353f5ba" (UID: "9faa6201-2130-402d-9408-acb9b353f5ba"). InnerVolumeSpecName "kube-api-access-8xgl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:39:07.875151 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.875124    2551 generic.go:356] "Generic (PLEG): container finished" podID="9faa6201-2130-402d-9408-acb9b353f5ba" containerID="5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8" exitCode=0
Sep 10 12:39:07.875257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.875176    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9"
Sep 10 12:39:07.875257 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.875196    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" event={"ID":"9faa6201-2130-402d-9408-acb9b353f5ba","Type":"ContainerDied","Data":"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"}
Sep 10 12:39:07.875395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.875277    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-qk9x9" event={"ID":"9faa6201-2130-402d-9408-acb9b353f5ba","Type":"ContainerDied","Data":"ce52601d4e6f1b042c331400127c704405d55c2372dc38fb3f606d215c04e1ba"}
Sep 10 12:39:07.875395 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.875311    2551 scope.go:117] "RemoveContainer" containerID="5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"
Sep 10 12:39:07.884349 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.884325    2551 scope.go:117] "RemoveContainer" containerID="5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"
Sep 10 12:39:07.884696 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:39:07.884670    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8\": container with ID starting with 5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8 not found: ID does not exist" containerID="5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"
Sep 10 12:39:07.884763 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.884702    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8"} err="failed to get container status \"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8\": rpc error: code = NotFound desc = could not find container \"5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8\": container with ID starting with 5e6c189082547d8af549ef3a4785e09eecf4f35d0262812e02862ed6b3a92cf8 not found: ID does not exist"
Sep 10 12:39:07.890042 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.890020    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xgl9\" (UniqueName: \"kubernetes.io/projected/9faa6201-2130-402d-9408-acb9b353f5ba-kube-api-access-8xgl9\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:39:07.890120 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.890046    2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:39:07.890120 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.890061    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa6201-2130-402d-9408-acb9b353f5ba-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:39:07.898727 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.898707    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:39:07.903088 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:07.903068    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-qk9x9"]
Sep 10 12:39:08.722833 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:08.722807    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9faa6201-2130-402d-9408-acb9b353f5ba" path="/var/lib/kubelet/pods/9faa6201-2130-402d-9408-acb9b353f5ba/volumes"
Sep 10 12:39:32.792151 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:32.792125    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:39:32.796192 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:32.796174    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:39:32.796355 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:32.796342    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:39:32.800438 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:39:32.800418    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:40:08.775411 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775289    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775707    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775721    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775740    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2b1b76a-cf06-4771-9588-fa33eef78a54" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775745    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2b1b76a-cf06-4771-9588-fa33eef78a54" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775753    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fff67c-b2b6-4f05-91a0-84009072c16a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775758    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fff67c-b2b6-4f05-91a0-84009072c16a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775764    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c061c338-e88c-40b5-8bf7-1e238151238b" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775769    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c061c338-e88c-40b5-8bf7-1e238151238b" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775776    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c354b2d2-723a-41ba-9322-6fea1210cfc4" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775781    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c354b2d2-723a-41ba-9322-6fea1210cfc4" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775787    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9faa6201-2130-402d-9408-acb9b353f5ba" containerName="manager"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775794    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9faa6201-2130-402d-9408-acb9b353f5ba" containerName="manager"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775856    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c061c338-e88c-40b5-8bf7-1e238151238b" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775863    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c354b2d2-723a-41ba-9322-6fea1210cfc4" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775871    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="62da0cf6-bf80-4bb5-92e8-b9b80190e94a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775878    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fff67c-b2b6-4f05-91a0-84009072c16a" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775887    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2b1b76a-cf06-4771-9588-fa33eef78a54" containerName="maas-api"
Sep 10 12:40:08.778582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.775896    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9faa6201-2130-402d-9408-acb9b353f5ba" containerName="manager"
Sep 10 12:40:08.779466 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.779336    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.788609 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.788589    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:40:08.789625 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.789609    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hmjln\""
Sep 10 12:40:08.789674 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.789632    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:40:08.789710 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.789632    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:40:08.861671 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.861638    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.861813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.861689    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg5rh\" (UniqueName: \"kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.861813 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.861710    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.893462 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.893425    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:08.962424 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.962305    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.962572 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.962486    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.962572 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.962552    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg5rh\" (UniqueName: \"kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.964788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.964763    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.964910 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.964802    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:08.979608 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:08.979577    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg5rh\" (UniqueName: \"kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh\") pod \"maas-controller-d6f6f5d54-67nhf\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") " pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:09.089121 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:09.089075    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:09.371070 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:40:09.371036    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18fc10f_d6f3_4ef4_ac2c_8043284a5e99.slice/crio-bb1d935f6d4e2bfb4965a8b73577f60073c778d1ce6ef42893fb461fcb17ec75 WatchSource:0}: Error finding container bb1d935f6d4e2bfb4965a8b73577f60073c778d1ce6ef42893fb461fcb17ec75: Status 404 returned error can't find the container with id bb1d935f6d4e2bfb4965a8b73577f60073c778d1ce6ef42893fb461fcb17ec75
Sep 10 12:40:09.371332 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:09.371313    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:09.372352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:09.372335    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:40:10.180921 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:10.180881    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" event={"ID":"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99","Type":"ContainerStarted","Data":"bb1d935f6d4e2bfb4965a8b73577f60073c778d1ce6ef42893fb461fcb17ec75"}
Sep 10 12:40:11.187242 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:11.187195    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" event={"ID":"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99","Type":"ContainerStarted","Data":"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"}
Sep 10 12:40:11.187691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:11.187410    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:11.227311 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:11.227258    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" podStartSLOduration=1.885363002 podStartE2EDuration="3.227233265s" totalImagesPullingTime="1.341870263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:40:11.225812999 +0000 UTC m=+1839.006370471" watchObservedRunningTime="2026-09-10 12:40:11.227233265 +0000 UTC m=+1839.007790815"
Sep 10 12:40:22.193885 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:22.193858    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:31.590188 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.590154    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:31.590582 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.590349    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" podUID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" containerName="manager" containerID="cri-o://e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc" gracePeriod=30
Sep 10 12:40:31.838251 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.838225    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:31.957596 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.957524    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs\") pod \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") "
Sep 10 12:40:31.957732 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.957602    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert\") pod \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") "
Sep 10 12:40:31.957732 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.957659    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg5rh\" (UniqueName: \"kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh\") pod \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\" (UID: \"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99\") "
Sep 10 12:40:31.959542 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.959508    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" (UID: "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:40:31.959542 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.959522    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" (UID: "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:40:31.959743 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:31.959582    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh" (OuterVolumeSpecName: "kube-api-access-sg5rh") pod "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" (UID: "d18fc10f-d6f3-4ef4-ac2c-8043284a5e99"). InnerVolumeSpecName "kube-api-access-sg5rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:40:32.058417 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.058395    2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:40:32.058417 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.058417    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg5rh\" (UniqueName: \"kubernetes.io/projected/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-kube-api-access-sg5rh\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:40:32.058558 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.058426    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:40:32.293691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.293634    2551 generic.go:356] "Generic (PLEG): container finished" podID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" containerID="e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc" exitCode=0
Sep 10 12:40:32.293691 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.293678    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" event={"ID":"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99","Type":"ContainerDied","Data":"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"}
Sep 10 12:40:32.293831 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.293689    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-67nhf"
Sep 10 12:40:32.293831 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.293719    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-67nhf" event={"ID":"d18fc10f-d6f3-4ef4-ac2c-8043284a5e99","Type":"ContainerDied","Data":"bb1d935f6d4e2bfb4965a8b73577f60073c778d1ce6ef42893fb461fcb17ec75"}
Sep 10 12:40:32.293831 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.293741    2551 scope.go:117] "RemoveContainer" containerID="e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"
Sep 10 12:40:32.302984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.302895    2551 scope.go:117] "RemoveContainer" containerID="e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"
Sep 10 12:40:32.303282 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:40:32.303262    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc\": container with ID starting with e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc not found: ID does not exist" containerID="e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"
Sep 10 12:40:32.303367 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.303292    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc"} err="failed to get container status \"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc\": rpc error: code = NotFound desc = could not find container \"e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc\": container with ID starting with e1ecb0f12214a4d4c6c99ecce794a3a012c7fb29616ad305065b9d136208c6dc not found: ID does not exist"
Sep 10 12:40:32.322366 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.322344    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:32.326337 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.326315    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-67nhf"]
Sep 10 12:40:32.726419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:32.726354    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" path="/var/lib/kubelet/pods/d18fc10f-d6f3-4ef4-ac2c-8043284a5e99/volumes"
Sep 10 12:40:33.168983 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.168951    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-j67nl"]
Sep 10 12:40:33.169331 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.169319    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" containerName="manager"
Sep 10 12:40:33.169393 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.169332    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" containerName="manager"
Sep 10 12:40:33.169438 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.169418    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18fc10f-d6f3-4ef4-ac2c-8043284a5e99" containerName="manager"
Sep 10 12:40:33.180235 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.180194    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.187733 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.187709    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:40:33.188044 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.187714    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hmjln\""
Sep 10 12:40:33.188316 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.187759    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:40:33.188425 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.187832    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:40:33.190795 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.190770    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-j67nl"]
Sep 10 12:40:33.268642 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.268618    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-webhook-cert\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.268745 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.268646    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-metrics-certs\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.268745 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.268703    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl8pc\" (UniqueName: \"kubernetes.io/projected/04198ede-dd90-4175-8c46-4605761dad92-kube-api-access-fl8pc\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.369954 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.369932    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-webhook-cert\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.370048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.369961    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-metrics-certs\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.370048 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.370030    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl8pc\" (UniqueName: \"kubernetes.io/projected/04198ede-dd90-4175-8c46-4605761dad92-kube-api-access-fl8pc\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.372409 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.372365    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-metrics-certs\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.372499 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.372434    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04198ede-dd90-4175-8c46-4605761dad92-webhook-cert\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.378977 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.378961    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl8pc\" (UniqueName: \"kubernetes.io/projected/04198ede-dd90-4175-8c46-4605761dad92-kube-api-access-fl8pc\") pod \"maas-controller-d6f6f5d54-j67nl\" (UID: \"04198ede-dd90-4175-8c46-4605761dad92\") " pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.493155 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.493098    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:33.766144 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:33.766114    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6f6f5d54-j67nl"]
Sep 10 12:40:33.767238 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:40:33.767214    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04198ede_dd90_4175_8c46_4605761dad92.slice/crio-85eecef05abafcf2755cd98a7ab7145426ae42e21f0ea61d24515636908acfcd WatchSource:0}: Error finding container 85eecef05abafcf2755cd98a7ab7145426ae42e21f0ea61d24515636908acfcd: Status 404 returned error can't find the container with id 85eecef05abafcf2755cd98a7ab7145426ae42e21f0ea61d24515636908acfcd
Sep 10 12:40:34.306210 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:34.306179    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-j67nl" event={"ID":"04198ede-dd90-4175-8c46-4605761dad92","Type":"ContainerStarted","Data":"80cfcac385028f2fbfdd926620e3efed7ce70eab4509b8cd3b2333fda05a33ef"}
Sep 10 12:40:34.306352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:34.306232    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6f6f5d54-j67nl" event={"ID":"04198ede-dd90-4175-8c46-4605761dad92","Type":"ContainerStarted","Data":"85eecef05abafcf2755cd98a7ab7145426ae42e21f0ea61d24515636908acfcd"}
Sep 10 12:40:34.306352 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:34.306303    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:40:34.334903 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:34.334857    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6f6f5d54-j67nl" podStartSLOduration=0.996477417 podStartE2EDuration="1.334844611s" totalImagesPullingTime="338.367194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:40:34.33242178 +0000 UTC m=+1862.112979271" watchObservedRunningTime="2026-09-10 12:40:34.334844611 +0000 UTC m=+1862.115402081"
Sep 10 12:40:45.313253 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:40:45.313224    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6f6f5d54-j67nl"
Sep 10 12:41:36.085509 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:36.085478    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:41:36.086036 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:36.085695    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" podUID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" containerName="manager" containerID="cri-o://391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4" gracePeriod=10
Sep 10 12:41:38.336699 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.336668    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:41:38.407330 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.407302    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume\") pod \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") "
Sep 10 12:41:38.407479 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.407404    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvmcz\" (UniqueName: \"kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz\") pod \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\" (UID: \"04166cbb-6445-4523-8d0d-9ce9af6fd81f\") "
Sep 10 12:41:38.407678 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.407654    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04166cbb-6445-4523-8d0d-9ce9af6fd81f" (UID: "04166cbb-6445-4523-8d0d-9ce9af6fd81f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:41:38.409329 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.409307    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz" (OuterVolumeSpecName: "kube-api-access-wvmcz") pod "04166cbb-6445-4523-8d0d-9ce9af6fd81f" (UID: "04166cbb-6445-4523-8d0d-9ce9af6fd81f"). InnerVolumeSpecName "kube-api-access-wvmcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:41:38.508298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.508243    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04166cbb-6445-4523-8d0d-9ce9af6fd81f-extensions-socket-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:41:38.508298 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.508263    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvmcz\" (UniqueName: \"kubernetes.io/projected/04166cbb-6445-4523-8d0d-9ce9af6fd81f-kube-api-access-wvmcz\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:41:38.627730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.627707    2551 generic.go:356] "Generic (PLEG): container finished" podID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" containerID="391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4" exitCode=0
Sep 10 12:41:38.627849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.627753    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"
Sep 10 12:41:38.627849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.627794    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" event={"ID":"04166cbb-6445-4523-8d0d-9ce9af6fd81f","Type":"ContainerDied","Data":"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"}
Sep 10 12:41:38.627849 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.627842    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9" event={"ID":"04166cbb-6445-4523-8d0d-9ce9af6fd81f","Type":"ContainerDied","Data":"9bde974c353d37c708f18317e64986a37ed8efd632b36357eb5ab3d1899ca95a"}
Sep 10 12:41:38.628014 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.627866    2551 scope.go:117] "RemoveContainer" containerID="391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"
Sep 10 12:41:38.638969 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.638944    2551 scope.go:117] "RemoveContainer" containerID="391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"
Sep 10 12:41:38.639448 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:41:38.639422    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4\": container with ID starting with 391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4 not found: ID does not exist" containerID="391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"
Sep 10 12:41:38.639541 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.639456    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4"} err="failed to get container status \"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4\": rpc error: code = NotFound desc = could not find container \"391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4\": container with ID starting with 391208e88a5262a590e661b04cee74a27ed4e50f13b94a7d9d8efa754d1f23a4 not found: ID does not exist"
Sep 10 12:41:38.652285 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.652263    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:41:38.656891 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.656870    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rblb9"]
Sep 10 12:41:38.722143 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:41:38.722120    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" path="/var/lib/kubelet/pods/04166cbb-6445-4523-8d0d-9ce9af6fd81f/volumes"
Sep 10 12:42:36.788978 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.788943    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"]
Sep 10 12:42:36.789504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.789306    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" containerName="manager"
Sep 10 12:42:36.789504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.789317    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" containerName="manager"
Sep 10 12:42:36.789504 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.789416    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="04166cbb-6445-4523-8d0d-9ce9af6fd81f" containerName="manager"
Sep 10 12:42:36.792262 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.792246    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.796707 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.796689    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zjzsx\""
Sep 10 12:42:36.827067 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.827046    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"]
Sep 10 12:42:36.836622 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.836602    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2m4v\" (UniqueName: \"kubernetes.io/projected/0de6a181-464e-44fb-ba63-6a8732857031-kube-api-access-c2m4v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.836717 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.836648    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de6a181-464e-44fb-ba63-6a8732857031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.937636 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.937608    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de6a181-464e-44fb-ba63-6a8732857031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.937769 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.937675    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2m4v\" (UniqueName: \"kubernetes.io/projected/0de6a181-464e-44fb-ba63-6a8732857031-kube-api-access-c2m4v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.937994 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.937971    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de6a181-464e-44fb-ba63-6a8732857031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:36.954430 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:36.954404    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2m4v\" (UniqueName: \"kubernetes.io/projected/0de6a181-464e-44fb-ba63-6a8732857031-kube-api-access-c2m4v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9z6js\" (UID: \"0de6a181-464e-44fb-ba63-6a8732857031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:37.102465 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.102441    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:37.385266 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.385235    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"]
Sep 10 12:42:37.385644 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:42:37.385624    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de6a181_464e_44fb_ba63_6a8732857031.slice/crio-7735cc891d60ef37660cf7b40f0c634a635e6cf944415979d12f574163bbe185 WatchSource:0}: Error finding container 7735cc891d60ef37660cf7b40f0c634a635e6cf944415979d12f574163bbe185: Status 404 returned error can't find the container with id 7735cc891d60ef37660cf7b40f0c634a635e6cf944415979d12f574163bbe185
Sep 10 12:42:37.910825 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.910792    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js" event={"ID":"0de6a181-464e-44fb-ba63-6a8732857031","Type":"ContainerStarted","Data":"1e49d9da00bdab57412a3195d403f51cad27ed665c9fbb4a96f749bec81ad3c2"}
Sep 10 12:42:37.911252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.910846    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js" event={"ID":"0de6a181-464e-44fb-ba63-6a8732857031","Type":"ContainerStarted","Data":"7735cc891d60ef37660cf7b40f0c634a635e6cf944415979d12f574163bbe185"}
Sep 10 12:42:37.911252 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.910908    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:42:37.939764 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:37.939721    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js" podStartSLOduration=1.9397092219999998 podStartE2EDuration="1.939709222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:42:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:42:37.939333361 +0000 UTC m=+1985.719890858" watchObservedRunningTime="2026-09-10 12:42:37.939709222 +0000 UTC m=+1985.720266694"
Sep 10 12:42:48.916974 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:42:48.916944    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9z6js"
Sep 10 12:44:32.822672 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:32.822647    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:44:32.827437 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:32.827414    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:44:32.827805 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:32.827789    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:44:32.832626 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:32.832607    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nj4hc_d4d415e4-d474-4315-a005-8f14ab9940f6/ovn-acl-logging/0.log"
Sep 10 12:44:45.304100 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.304058    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"]
Sep 10 12:44:45.308109 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.308088    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.310825 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.310807    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-89jb9\""
Sep 10 12:44:45.320901 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.320876    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"]
Sep 10 12:44:45.332937 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.332916    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-maas-api-tls\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.333045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.332984    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljjnh\" (UniqueName: \"kubernetes.io/projected/06ad5137-3a37-42c3-8b04-92df5549f8cb-kube-api-access-ljjnh\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.333045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.333015    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-metrics-certs\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.333045 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.333033    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad5137-3a37-42c3-8b04-92df5549f8cb-openshift-service-ca\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.434281 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.434236    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljjnh\" (UniqueName: \"kubernetes.io/projected/06ad5137-3a37-42c3-8b04-92df5549f8cb-kube-api-access-ljjnh\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.434457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.434293    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-metrics-certs\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.434457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.434324    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad5137-3a37-42c3-8b04-92df5549f8cb-openshift-service-ca\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.434457 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.434442    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-maas-api-tls\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.437452 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.435671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad5137-3a37-42c3-8b04-92df5549f8cb-openshift-service-ca\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.441740 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.437953    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-maas-api-tls\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.441740 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.438024    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ad5137-3a37-42c3-8b04-92df5549f8cb-metrics-certs\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.444407 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.444368    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljjnh\" (UniqueName: \"kubernetes.io/projected/06ad5137-3a37-42c3-8b04-92df5549f8cb-kube-api-access-ljjnh\") pod \"maas-api-6fcd467ffd-xdx6f\" (UID: \"06ad5137-3a37-42c3-8b04-92df5549f8cb\") " pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.620991 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.620962    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:45.894726 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:45.894633    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"]
Sep 10 12:44:45.897692 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:44:45.897667    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ad5137_3a37_42c3_8b04_92df5549f8cb.slice/crio-aeb45572c0004a3b440d49de047a43a9324334a63ecbe1375fa49f1a98ce7d53 WatchSource:0}: Error finding container aeb45572c0004a3b440d49de047a43a9324334a63ecbe1375fa49f1a98ce7d53: Status 404 returned error can't find the container with id aeb45572c0004a3b440d49de047a43a9324334a63ecbe1375fa49f1a98ce7d53
Sep 10 12:44:46.533525 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:46.533493    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f" event={"ID":"06ad5137-3a37-42c3-8b04-92df5549f8cb","Type":"ContainerStarted","Data":"85443a1dfe76bb71f46131e735d4cff75c4eac9068bb1b8b7f8d7147e1242686"}
Sep 10 12:44:46.533847 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:46.533546    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f" event={"ID":"06ad5137-3a37-42c3-8b04-92df5549f8cb","Type":"ContainerStarted","Data":"aeb45572c0004a3b440d49de047a43a9324334a63ecbe1375fa49f1a98ce7d53"}
Sep 10 12:44:46.533847 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:46.533570    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:44:46.559592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:46.559544    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f" podStartSLOduration=1.118233316 podStartE2EDuration="1.559527836s" totalImagesPullingTime="441.29452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:44:46.557659441 +0000 UTC m=+2114.338216925" watchObservedRunningTime="2026-09-10 12:44:46.559527836 +0000 UTC m=+2114.340085309"
Sep 10 12:44:52.544001 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:44:52.543972    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6fcd467ffd-xdx6f"
Sep 10 12:45:01.720837 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:01.720802    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:45:01.721331 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:01.721017    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerName="maas-api" containerID="cri-o://552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70" gracePeriod=30
Sep 10 12:45:02.446544 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:02.446505    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.49:8443/health\": dial tcp 10.132.0.49:8443: connect: connection refused"
Sep 10 12:45:02.971263 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:02.971240    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:45:03.076358 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076288    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca\") pod \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") "
Sep 10 12:45:03.076358 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076357    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g6nr\" (UniqueName: \"kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr\") pod \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") "
Sep 10 12:45:03.076533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076487    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls\") pod \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") "
Sep 10 12:45:03.076533 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076517    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") pod \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\" (UID: \"f1da36ce-144f-4e72-a4ed-06724ae8bd7e\") "
Sep 10 12:45:03.076742 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076710    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1da36ce-144f-4e72-a4ed-06724ae8bd7e" (UID: "f1da36ce-144f-4e72-a4ed-06724ae8bd7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:45:03.076837 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.076796    2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:45:03.078503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.078476    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1da36ce-144f-4e72-a4ed-06724ae8bd7e" (UID: "f1da36ce-144f-4e72-a4ed-06724ae8bd7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:45:03.078594 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.078567    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr" (OuterVolumeSpecName: "kube-api-access-4g6nr") pod "f1da36ce-144f-4e72-a4ed-06724ae8bd7e" (UID: "f1da36ce-144f-4e72-a4ed-06724ae8bd7e"). InnerVolumeSpecName "kube-api-access-4g6nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:45:03.078678 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.078660    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1da36ce-144f-4e72-a4ed-06724ae8bd7e" (UID: "f1da36ce-144f-4e72-a4ed-06724ae8bd7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:45:03.177923 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.177898    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g6nr\" (UniqueName: \"kubernetes.io/projected/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-kube-api-access-4g6nr\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:45:03.177923 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.177920    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:45:03.178051 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.177929    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1da36ce-144f-4e72-a4ed-06724ae8bd7e-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\""
Sep 10 12:45:03.620506 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.620467    2551 generic.go:356] "Generic (PLEG): container finished" podID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerID="552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70" exitCode=0
Sep 10 12:45:03.620664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.620521    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78"
Sep 10 12:45:03.620664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.620542    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" event={"ID":"f1da36ce-144f-4e72-a4ed-06724ae8bd7e","Type":"ContainerDied","Data":"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"}
Sep 10 12:45:03.620664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.620608    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cc544454-6kg78" event={"ID":"f1da36ce-144f-4e72-a4ed-06724ae8bd7e","Type":"ContainerDied","Data":"d7131222d5fd6391c1f2b2c32ca034b1a2ab22f386a10b3860928085f91c8887"}
Sep 10 12:45:03.620664 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.620632    2551 scope.go:117] "RemoveContainer" containerID="552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"
Sep 10 12:45:03.630204 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.630187    2551 scope.go:117] "RemoveContainer" containerID="552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"
Sep 10 12:45:03.630495 ip-10-0-130-59 kubenswrapper[2551]: E0910 12:45:03.630476    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70\": container with ID starting with 552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70 not found: ID does not exist" containerID="552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"
Sep 10 12:45:03.630541 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.630518    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70"} err="failed to get container status \"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70\": rpc error: code = NotFound desc = could not find container \"552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70\": container with ID starting with 552e0bffbf3cd58d378d6db170f26a14a8f901d17935d1659a3dc489497ebf70 not found: ID does not exist"
Sep 10 12:45:03.643271 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.643251    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:45:03.648687 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:03.648666    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cc544454-6kg78"]
Sep 10 12:45:04.722296 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:04.722265    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" path="/var/lib/kubelet/pods/f1da36ce-144f-4e72-a4ed-06724ae8bd7e/volumes"
Sep 10 12:45:15.025027 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:15.024998    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b694d445c-g8kmn_7ff6c168-88f2-49a5-b3ee-0ab8253f0a00/authorino/0.log"
Sep 10 12:45:20.205025 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:20.204943    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-djwfr_28a25d9d-363f-4bb2-9770-aa511b9fca5a/manager/0.log"
Sep 10 12:45:20.494014 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:20.493947    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6f6f5d54-j67nl_04198ede-dd90-4175-8c46-4605761dad92/manager/0.log"
Sep 10 12:45:20.826310 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:20.826253    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f75798bc7-fkpq6_5379cddf-005d-477e-bed8-069b0d5d7e25/copy-manifests/0.log"
Sep 10 12:45:20.844047 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:20.844024    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f75798bc7-fkpq6_5379cddf-005d-477e-bed8-069b0d5d7e25/manager/0.log"
Sep 10 12:45:21.512862 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:21.512837    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6fcd467ffd-xdx6f_06ad5137-3a37-42c3-8b04-92df5549f8cb/maas-api/0.log"
Sep 10 12:45:21.848329 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:21.848308    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7ng6x_5dd7764b-e247-4a5c-b862-b2c0b093e348/postgres/0.log"
Sep 10 12:45:22.582180 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:22.582156    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b694d445c-g8kmn_7ff6c168-88f2-49a5-b3ee-0ab8253f0a00/authorino/0.log"
Sep 10 12:45:22.694788 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:22.694760    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fwfsv_47f123ed-b819-45f6-aab7-0b3e019e4ae1/manager/0.log"
Sep 10 12:45:22.808246 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:22.808222    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-cnm8k_40b253a3-dc01-459b-b4b4-9653b799491b/manager/0.log"
Sep 10 12:45:23.043230 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:23.043170    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-9z6js_0de6a181-464e-44fb-ba63-6a8732857031/manager/0.log"
Sep 10 12:45:23.272049 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:23.272022    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jwxxq_773b95ef-0eef-411d-88e9-7688a1f71055/manager/0.log"
Sep 10 12:45:23.839324 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:23.839298    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688cb69f77-9llgl_07a8d33f-cdf4-48ef-9a98-650a2be25e02/kube-auth-proxy/0.log"
Sep 10 12:45:24.193592 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:24.193530    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ckqsj_016c941a-48e6-4de3-bead-b8e8f9fe7678/payload-pre-processing/0.log"
Sep 10 12:45:24.305339 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:24.305314    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:45:24.426317 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:24.426295    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b6767dd58-vrxv6_b45f74b2-bcbd-4d26-881f-ff82ec49c1eb/router/0.log"
Sep 10 12:45:24.976610 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:24.976550    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-s4dq5_efaf8909-40e0-4329-ab3c-eca26be6a5b8/main/0.log"
Sep 10 12:45:25.312645 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:25.312590    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-wpgf6_1e346e75-bad3-4e80-be42-5b47985c1de7/main/0.log"
Sep 10 12:45:34.255464 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:34.255431    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x9z4k_d86fe67c-88f7-4a0f-b640-c1150019d3fc/global-pull-secret-syncer/0.log"
Sep 10 12:45:34.435565 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:34.435540    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wcjsq_61c014f8-7c3a-4f98-8602-79e3286d8604/konnectivity-agent/0.log"
Sep 10 12:45:34.488940 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:34.488906    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-59.ec2.internal_48adfbd41af55e45ba98e8777ebfc093/haproxy/0.log"
Sep 10 12:45:38.865986 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:38.865956    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b694d445c-g8kmn_7ff6c168-88f2-49a5-b3ee-0ab8253f0a00/authorino/0.log"
Sep 10 12:45:38.896194 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:38.896165    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fwfsv_47f123ed-b819-45f6-aab7-0b3e019e4ae1/manager/0.log"
Sep 10 12:45:38.921287 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:38.921267    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-cnm8k_40b253a3-dc01-459b-b4b4-9653b799491b/manager/0.log"
Sep 10 12:45:38.997930 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:38.997906    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-9z6js_0de6a181-464e-44fb-ba63-6a8732857031/manager/0.log"
Sep 10 12:45:39.122819 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:39.122761    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jwxxq_773b95ef-0eef-411d-88e9-7688a1f71055/manager/0.log"
Sep 10 12:45:40.713583 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:40.713551    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-krbld_b0ec98c0-424b-4746-a9d9-ba63f1eb343b/cluster-monitoring-operator/0.log"
Sep 10 12:45:41.036730 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:41.036668    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6djt_8bff6cd3-776f-4b23-8e91-e8ccdf50cd43/node-exporter/0.log"
Sep 10 12:45:41.083597 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:41.083575    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6djt_8bff6cd3-776f-4b23-8e91-e8ccdf50cd43/kube-rbac-proxy/0.log"
Sep 10 12:45:41.128095 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:41.128077    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6djt_8bff6cd3-776f-4b23-8e91-e8ccdf50cd43/init-textfile/0.log"
Sep 10 12:45:41.654419 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:41.654398    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-72glj_caed5e87-fec1-4182-b625-2ea886deba82/prometheus-operator-admission-webhook/0.log"
Sep 10 12:45:43.226651 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.226621    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-b8kql_08296382-68e4-4fa4-837b-47b37de1c881/networking-console-plugin/0.log"
Sep 10 12:45:43.729984 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.729953    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"]
Sep 10 12:45:43.730405 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.730388    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerName="maas-api"
Sep 10 12:45:43.730503 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.730406    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerName="maas-api"
Sep 10 12:45:43.730560 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.730509    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1da36ce-144f-4e72-a4ed-06724ae8bd7e" containerName="maas-api"
Sep 10 12:45:43.733415 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.733398    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.736773 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.736757    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-htg6h\"/\"default-dockercfg-cc8xr\""
Sep 10 12:45:43.737471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.737454    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-htg6h\"/\"openshift-service-ca.crt\""
Sep 10 12:45:43.737541 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.737460    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-htg6h\"/\"kube-root-ca.crt\""
Sep 10 12:45:43.760881 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.760852    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"]
Sep 10 12:45:43.792890 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.792867    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwmdn\" (UniqueName: \"kubernetes.io/projected/811a7388-bc6c-41c9-a056-8189f5ff2d0e-kube-api-access-vwmdn\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.792992 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.792915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-sys\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.793057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.792994    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-proc\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.793057 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.793039    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-lib-modules\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.793164 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.793068    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-podres\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.893883 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.893849    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwmdn\" (UniqueName: \"kubernetes.io/projected/811a7388-bc6c-41c9-a056-8189f5ff2d0e-kube-api-access-vwmdn\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.893980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.893899    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-sys\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.893980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.893923    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-proc\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.893980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.893944    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-lib-modules\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.893980 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.893960    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-podres\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.894134 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.894027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-proc\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.894134 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.894060    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-sys\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.894134 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.894102    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-podres\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.894134 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.894102    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/811a7388-bc6c-41c9-a056-8189f5ff2d0e-lib-modules\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.903440 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.903412    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/1.log"
Sep 10 12:45:43.907442 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.907421    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwmdn\" (UniqueName: \"kubernetes.io/projected/811a7388-bc6c-41c9-a056-8189f5ff2d0e-kube-api-access-vwmdn\") pod \"perf-node-gather-daemonset-852lj\" (UID: \"811a7388-bc6c-41c9-a056-8189f5ff2d0e\") " pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:43.912056 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:43.912031    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zlph5_2d29e9a0-2c23-42b5-80b1-2a961a66e48e/console-operator/2.log"
Sep 10 12:45:44.043203 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.043153    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:44.320729 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.320707    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"]
Sep 10 12:45:44.321949 ip-10-0-130-59 kubenswrapper[2551]: W0910 12:45:44.321925    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod811a7388_bc6c_41c9_a056_8189f5ff2d0e.slice/crio-c4414ed5366abbaf99e14427edd204b9f86f1917716969eadcec12275c80555e WatchSource:0}: Error finding container c4414ed5366abbaf99e14427edd204b9f86f1917716969eadcec12275c80555e: Status 404 returned error can't find the container with id c4414ed5366abbaf99e14427edd204b9f86f1917716969eadcec12275c80555e
Sep 10 12:45:44.490765 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.490741    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86845cf594-bghc4_b0197622-b527-43c2-a846-7c03c54fa12e/console/0.log"
Sep 10 12:45:44.529562 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.529544    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-sg47x_ff2fb597-34ab-4a41-914a-1c6262f54efb/download-server/0.log"
Sep 10 12:45:44.823626 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.823601    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj" event={"ID":"811a7388-bc6c-41c9-a056-8189f5ff2d0e","Type":"ContainerStarted","Data":"45a96f71d5d6a0cb9cbc6233a8f70ca31ae779c394189c3f03a0ec012ec79b57"}
Sep 10 12:45:44.823758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.823651    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj" event={"ID":"811a7388-bc6c-41c9-a056-8189f5ff2d0e","Type":"ContainerStarted","Data":"c4414ed5366abbaf99e14427edd204b9f86f1917716969eadcec12275c80555e"}
Sep 10 12:45:44.823758 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.823749    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:44.843293 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:44.843255    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj" podStartSLOduration=1.843243693 podStartE2EDuration="1.843243693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:45:44.842017101 +0000 UTC m=+2172.622574594" watchObservedRunningTime="2026-09-10 12:45:44.843243693 +0000 UTC m=+2172.623801198"
Sep 10 12:45:45.212961 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:45.212939    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-ttt4x_0f1a4986-32cd-45f9-86b6-3eb5410b7069/volume-data-source-validator/0.log"
Sep 10 12:45:46.365128 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:46.365074    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-29d4k_5208d4e0-3d95-49be-80f3-f3e681fc63b5/dns/0.log"
Sep 10 12:45:46.392815 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:46.392773    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-29d4k_5208d4e0-3d95-49be-80f3-f3e681fc63b5/kube-rbac-proxy/0.log"
Sep 10 12:45:46.559979 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:46.559960    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-569pc_955654cc-3b3c-4320-bb69-76ba876dc342/dns-node-resolver/0.log"
Sep 10 12:45:47.185979 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:47.185942    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9w79l_5f70ccae-062e-4a1d-8191-20e3b9e49c33/node-ca/0.log"
Sep 10 12:45:48.498506 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:48.498481    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688cb69f77-9llgl_07a8d33f-cdf4-48ef-9a98-650a2be25e02/kube-auth-proxy/0.log"
Sep 10 12:45:48.626003 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:48.625970    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ckqsj_016c941a-48e6-4de3-bead-b8e8f9fe7678/payload-pre-processing/0.log"
Sep 10 12:45:48.668476 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:48.668442    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:45:48.700072 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:48.700037    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b6767dd58-vrxv6_b45f74b2-bcbd-4d26-881f-ff82ec49c1eb/router/0.log"
Sep 10 12:45:49.643900 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:49.643868    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lrtv7_82423b91-84fc-44e5-b3a3-5f3c23ee4452/serve-healthcheck-canary/0.log"
Sep 10 12:45:50.178471 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.178444    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-tgczq_40af8ac6-a203-4ac5-8e54-64baac884dda/insights-operator/0.log"
Sep 10 12:45:50.180896 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.180868    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-tgczq_40af8ac6-a203-4ac5-8e54-64baac884dda/insights-operator/1.log"
Sep 10 12:45:50.371078 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.371054    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlkgc_e3906a31-caa0-4235-acc1-dd77ad2624f0/kube-rbac-proxy/0.log"
Sep 10 12:45:50.398458 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.398439    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlkgc_e3906a31-caa0-4235-acc1-dd77ad2624f0/exporter/0.log"
Sep 10 12:45:50.430450 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.430401    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlkgc_e3906a31-caa0-4235-acc1-dd77ad2624f0/extractor/0.log"
Sep 10 12:45:50.838122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:50.838104    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-htg6h/perf-node-gather-daemonset-852lj"
Sep 10 12:45:52.820105 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:52.820082    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-djwfr_28a25d9d-363f-4bb2-9770-aa511b9fca5a/manager/0.log"
Sep 10 12:45:53.105834 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:53.105806    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6f6f5d54-j67nl_04198ede-dd90-4175-8c46-4605761dad92/manager/0.log"
Sep 10 12:45:53.332818 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:53.332792    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f75798bc7-fkpq6_5379cddf-005d-477e-bed8-069b0d5d7e25/manager/0.log"
Sep 10 12:45:53.356221 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:53.356167    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f75798bc7-fkpq6_5379cddf-005d-477e-bed8-069b0d5d7e25/copy-manifests/0.log"
Sep 10 12:45:54.087698 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:54.087671    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688cb69f77-9llgl_07a8d33f-cdf4-48ef-9a98-650a2be25e02/kube-auth-proxy/0.log"
Sep 10 12:45:54.170540 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:54.170516    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ckqsj_016c941a-48e6-4de3-bead-b8e8f9fe7678/payload-pre-processing/0.log"
Sep 10 12:45:54.208842 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:54.208820    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:45:54.227363 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:54.227339    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b6767dd58-vrxv6_b45f74b2-bcbd-4d26-881f-ff82ec49c1eb/router/0.log"
Sep 10 12:45:55.935069 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:55.935039    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688cb69f77-9llgl_07a8d33f-cdf4-48ef-9a98-650a2be25e02/kube-auth-proxy/0.log"
Sep 10 12:45:56.026816 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:56.026791    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ckqsj_016c941a-48e6-4de3-bead-b8e8f9fe7678/payload-pre-processing/0.log"
Sep 10 12:45:56.063152 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:56.063132    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zpkr4_8fc82292-8457-437d-89d5-6b0cc827bc4f/payload-processing/0.log"
Sep 10 12:45:56.084122 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:56.084079    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b6767dd58-vrxv6_b45f74b2-bcbd-4d26-881f-ff82ec49c1eb/router/0.log"
Sep 10 12:45:59.610766 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:59.610727    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-rhrds_0ef0240a-a279-4c92-a8d3-e7ef83b5aae4/migrator/0.log"
Sep 10 12:45:59.634902 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:59.634877    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-rhrds_0ef0240a-a279-4c92-a8d3-e7ef83b5aae4/graceful-termination/0.log"
Sep 10 12:45:59.998958 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:45:59.998885    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-jxnnl_90fb8de4-a0c3-4250-bca8-44c17293d423/kube-storage-version-migrator-operator/1.log"
Sep 10 12:46:00.000316 ip-10-0-130-59 kubenswrapper[2551]: I0910 12:46:00.000294    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-jxnnl_90fb8de4-a0c3-4250-bca8-44c17293d423/kube-storage-version-migrator-operator/0.log"