Jul 06 19:48:21.625393 ip-10-0-132-142 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 19:48:22.437259 ip-10-0-132-142 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:48:22.437259 ip-10-0-132-142 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 19:48:22.437259 ip-10-0-132-142 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:48:22.437259 ip-10-0-132-142 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 19:48:22.437259 ip-10-0-132-142 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:48:22.437966 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.437273    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442601    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442620    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442626    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442629    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442633    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.442629 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442637    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442640    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442644    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442647    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442649    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442653    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442656    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442658    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442662    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442665    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442667    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442670    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442673    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442676    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442679    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442682    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442685    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442687    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442691    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.442868 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442694    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442699    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442704    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442707    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442711    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442714    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442718    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442720    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442723    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442726    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442729    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442732    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442735    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442738    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442740    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442744    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442747    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442749    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442752    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.443356 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442755    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442758    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442761    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442764    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442766    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442771    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442774    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442778    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442780    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442784    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442787    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442790    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442793    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442797    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442801    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442804    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442807    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442810    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442813    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.443841 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442816    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442818    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442821    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442826    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442829    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442833    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442836    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442839    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442841    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442846    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442851    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442854    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442857    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442859    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442862    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442865    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442868    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442870    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442874    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442877    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.444340 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442880    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442884    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442886    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442889    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442894    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442897    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442899    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442902    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442905    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442908    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442911    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442914    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442917    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442920    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442923    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442926    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.442928    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443706    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443714    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443717    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443720    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.444835 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443723    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443726    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443729    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443732    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443735    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443737    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443740    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443742    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443745    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443748    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443751    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443754    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443757    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443761    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443764    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443767    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443769    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443772    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443774    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443777    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.445366 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443780    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443783    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443785    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443788    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443791    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443793    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443796    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443798    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443802    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443806    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443810    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443813    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443816    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443818    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443821    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443824    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443826    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443829    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443832    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.445889 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443835    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443837    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443840    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443843    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443846    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443849    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443852    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443856    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443859    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443862    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443864    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443867    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443870    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443872    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443875    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443878    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443880    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443883    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443886    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443889    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.446403 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443894    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443898    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443901    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443904    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443907    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443911    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443914    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443917    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443920    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443923    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443926    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443929    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443932    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443934    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443937    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443940    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443942    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443945    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443948    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443951    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.446914 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443955    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443957    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443960    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443965    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443969    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443973    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443976    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443979    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.443982    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444002    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444006    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444009    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444012    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444014    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444017    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444020    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444405    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444410    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.447479 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444413    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444417    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444420    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444423    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444426    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444428    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444431    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444434    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444437    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444440    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444443    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444446    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444448    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444451    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444454    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444457    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444460    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444463    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444465    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.447954 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444468    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444471    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444473    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444476    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444479    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444481    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444484    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444487    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444490    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444492    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444495    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444498    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444502    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444505    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444508    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444511    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444514    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444516    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444519    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444521    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.448449 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444524    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444527    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444529    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444532    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444535    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444537    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444540    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444543    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444546    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444549    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444551    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444554    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444557    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444559    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444562    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444564    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444567    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444570    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444572    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444576    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.448996 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444578    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444581    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444585    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444587    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444592    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444596    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444600    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444602    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444605    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444609    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444613    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444616    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444619    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444622    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444625    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444628    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444630    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444633    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444636    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.449508 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444638    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444642    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444645    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444648    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444650    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444653    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444656    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444658    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444661    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444664    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444666    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444671    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444674    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444678    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444681    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444684    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444687    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444692    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.444695    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444767    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 19:48:22.450049 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444774    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444780    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444785    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444789    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444792    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444797    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444801    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444804    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444807    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444811    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444814    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444818    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444822    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444824    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444828    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444831    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444834    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444838    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444842    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444845    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444848    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444852    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444856    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 19:48:22.450606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444859    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444862    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444865    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444868    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444872    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444875    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444880    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444884    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444888    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444891    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444894    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444897    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444900    2565 flags.go:64] FLAG: --enable-server="true"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444903    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444908    2565 flags.go:64] FLAG: --event-burst="100"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444911    2565 flags.go:64] FLAG: --event-qps="50"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444914    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444917    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444921    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444924    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444928    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444931    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444934    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444937    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444940    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 19:48:22.451181 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444944    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444949    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444952    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444955    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444958    2565 flags.go:64] FLAG: --feature-gates=""
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444962    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444965    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444969    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444972    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444975    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444978    2565 flags.go:64] FLAG: --help="false"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.444981    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445001    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445007    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445010    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445013    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445017    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445020    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445023    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445026    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445029    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445032    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445036    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445039    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 06 19:48:22.451837 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445042    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445045    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445048    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445051    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445054    2565 flags.go:64] FLAG: --lock-file=""
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445057    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445060    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445063    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445069    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445074    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445077    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445080    2565 flags.go:64] FLAG: --logging-format="text"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445083    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445086    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445089    2565 flags.go:64] FLAG: --manifest-url=""
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445092    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445102    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445105    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445109    2565 flags.go:64] FLAG: --max-pods="110"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445112    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445115    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445120    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445123    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445126    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445129    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 19:48:22.452567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445132    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445139    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445143    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445146    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445149    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445152    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445158    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445161    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445164    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445168    2565 flags.go:64] FLAG: --port="10250"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445171    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445174    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d2b3e17e6cbac440"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445177    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445180    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445183    2565 flags.go:64] FLAG: --register-node="true"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445186    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445191    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445195    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445198    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445201    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445205    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445208    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445211    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445214    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445217    2565 flags.go:64] FLAG: --runonce="false"
Jul 06 19:48:22.453240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445220    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445223    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445227    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445229    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445234    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445237    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445240    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445243    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445246    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445249    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445252    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445255    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445258    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445261    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445264    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445269    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445272    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445275    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445279    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445282    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445285    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445288    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445291    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445296    2565 flags.go:64] FLAG: --v="2"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445300    2565 flags.go:64] FLAG: --version="false"
Jul 06 19:48:22.453879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445308    2565 flags.go:64] FLAG: --vmodule=""
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445312    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.445315    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445435    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445439    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445442    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445446    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445449    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445452    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445455    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445458    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445462    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445465    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445468    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445471    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445474    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445477    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445479    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445482    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445485    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.454567 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445488    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445491    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445493    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445496    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445499    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445502    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445505    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445508    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445512    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445516    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445520    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445524    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445528    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445532    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445535    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445537    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445540    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445543    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445545    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.455164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445548    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445551    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445554    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445556    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445560    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445563    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445566    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445569    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445571    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445574    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445577    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445580    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445583    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445585    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445588    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445591    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445593    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445596    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445599    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.455662 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445602    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445605    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445607    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445610    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445615    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445618    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445621    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445623    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445626    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445629    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445632    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445634    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445637    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445640    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445643    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445645    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445648    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445652    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445655    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445657    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.456200 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445660    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445663    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445666    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445668    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445671    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445674    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445676    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445679    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445682    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445685    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445688    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445691    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445693    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445696    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445698    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445701    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445705    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445708    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445710    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.456710 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445714    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445718    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445722    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445725    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.445728    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.446771    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.453770    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.453787    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.454517    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.454526    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454585    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454590    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454594    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454597    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454601    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.457213 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454604    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454607    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454610    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454613    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454616    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454619    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454622    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454625    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454640    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454644    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454647    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454650    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454653    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454656    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454658    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454661    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454664    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454667    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454669    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454673    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.457602 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454676    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454679    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454682    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454685    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454688    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454691    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454694    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454697    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454699    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454702    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454708    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454712    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454715    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454718    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454721    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454724    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454727    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454730    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454733    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.458149 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454736    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454739    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454742    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454745    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454747    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454750    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454753    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454757    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454760    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454764    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454767    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454770    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454774    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454777    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454780    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454783    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454786    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454790    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454792    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454795    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.458674 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454798    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454801    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454803    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454806    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454809    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454811    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454814    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454817    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454821    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454824    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454827    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454830    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454832    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454835    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454837    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454840    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454843    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454846    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454848    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454851    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.459295 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454854    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454857    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454860    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454862    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454865    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454868    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454871    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454874    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454876    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454880    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454883    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454885    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454888    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454891    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.454894    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.459810 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.454899    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455066    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455072    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455076    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455079    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455083    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455086    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455089    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455092    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455095    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455099    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455102    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455105    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455108    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455110    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455113    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455116    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455119    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455122    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:48:22.460270 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455125    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455127    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455130    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455133    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455136    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455141    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455145    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455148    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455152    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455155    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455158    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455161    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455164    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455168    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455171    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455174    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455176    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455179    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455182    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455185    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:48:22.460957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455188    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455190    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455193    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455196    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455198    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455201    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455204    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455207    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455209    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455212    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455215    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455218    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455220    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455223    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455226    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455228    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455231    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455234    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455237    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455240    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:48:22.461515 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455243    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455246    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455249    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455251    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455254    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455257    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455260    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455262    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455265    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455269    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455273    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455276    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455278    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455281    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455284    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455287    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455290    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455292    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455295    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455298    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:48:22.462052 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455301    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455305    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455307    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455310    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455313    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455315    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455318    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455321    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455324    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455327    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455329    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455333    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455336    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455339    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455341    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455345    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455348    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455351    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455353    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:48:22.462816 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455356    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:22.455359    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.455364    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.455468    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.458398    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.460297    2565 server.go:1019] "Starting client certificate rotation"
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.460447    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.460514    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 19:48:22.463466 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.463050    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 19:48:22.464903 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.464880    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 19:48:22.477469 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.477452    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 06 19:48:22.481039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.480903    2565 log.go:25] "Validated CRI v1 image API"
Jul 06 19:48:22.482150 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.482132    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 19:48:22.484692 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.484661    2565 fs.go:135] Filesystem UUIDs: map[19cfa22f-5cb7-4003-84e0-c59f2da0edd3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d197a35f-b2c9-44a9-902b-0f944bf3b84d:/dev/nvme0n1p3]
Jul 06 19:48:22.484773 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.484688    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 19:48:22.491202 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.491083    2565 manager.go:216] Machine: {Timestamp:2026-07-06 19:48:22.489760304 +0000 UTC m=+0.751739162 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100028 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26451b36ccd5ff3924876fbe71e3e1 SystemUUID:ec26451b-36cc-d5ff-3924-876fbe71e3e1 BootID:deee71e5-edaa-4bc1-9f76-72289c23f127 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:53:d1:46:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:53:d1:46:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:d0:fe:a0:0e:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 19:48:22.491202 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.491191    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 19:48:22.491343 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.491310    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 19:48:22.493110 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.493093    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 19:48:22.494487 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.494457    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 19:48:22.494645 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.494491    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 19:48:22.495844 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.495831    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 19:48:22.495879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.495848    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 19:48:22.495879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.495856    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 19:48:22.496468 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.496457    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 19:48:22.497799 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.497789    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 19:48:22.500712 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500699    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 19:48:22.500836 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500826    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 19:48:22.500881 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500856    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 19:48:22.500881 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500870    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 19:48:22.501023 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500892    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 19:48:22.501023 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500905    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 06 19:48:22.501023 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.500916    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 19:48:22.501407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.501391    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 19:48:22.501482 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.501418    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 19:48:22.503627 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.503608    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 19:48:22.505557 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.505542    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 19:48:22.505600 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.505577    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 19:48:22.506680 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506664    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506686    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506712    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506723    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506733    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506739    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 19:48:22.506747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506748    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 19:48:22.507013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506753    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 19:48:22.507013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506761    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 19:48:22.507013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506766    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 19:48:22.507013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506782    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 19:48:22.507013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.506787    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 19:48:22.507693 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.507682    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 19:48:22.507693 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.507694    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 19:48:22.511311 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.511295    2565 server.go:1268] "Started kubelet"
Jul 06 19:48:22.511565 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.511528    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 19:48:22.512274 ip-10-0-132-142 systemd[1]: Started Kubernetes Kubelet.
Jul 06 19:48:22.512801 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.511898    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 19:48:22.512801 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.512444    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 19:48:22.512801 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.512544    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 19:48:22.512801 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.512541    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 19:48:22.512801 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.512621    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 19:48:22.516264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.516164    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 19:48:22.518249 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.518209    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 19:48:22.519153 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.519139    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 19:48:22.519736 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.519716    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s85s4"
Jul 06 19:48:22.520600 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.518452    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-142.ec2.internal.18bfcb1cea3b002f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-142.ec2.internal,UID:ip-10-0-132-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-142.ec2.internal,},FirstTimestamp:2026-07-06 19:48:22.511263791 +0000 UTC m=+0.773242657,LastTimestamp:2026-07-06 19:48:22.511263791 +0000 UTC m=+0.773242657,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-142.ec2.internal,}"
Jul 06 19:48:22.522877 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.522862    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 19:48:22.522934 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.522876    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 19:48:22.523016 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.522977    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523582    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523702    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523721    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523781    2565 factory.go:55] Registering systemd factory
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523815    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523863    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 19:48:22.523867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.523871    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 19:48:22.524267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524232    2565 factory.go:153] Registering CRI-O factory
Jul 06 19:48:22.524267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524247    2565 factory.go:223] Registration of the crio container factory successfully
Jul 06 19:48:22.524371 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524302    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 19:48:22.524371 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524329    2565 factory.go:103] Registering Raw factory
Jul 06 19:48:22.524371 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524346    2565 manager.go:1196] Started watching for new ooms in manager
Jul 06 19:48:22.524867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.524850    2565 manager.go:349] Starting recovery of all containers
Jul 06 19:48:22.527671 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.527650    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s85s4"
Jul 06 19:48:22.534129 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.534108    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:22.536977 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.536951    2565 manager.go:354] Recovery completed
Jul 06 19:48:22.537104 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.537087    2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-142.ec2.internal\" not found" node="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.541314 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.541287    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.543598 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.543581    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.543678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.543613    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.543678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.543626    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.544213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.544198    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 19:48:22.544213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.544210    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 19:48:22.544356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.544216    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.544356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.544225    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 19:48:22.545339 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545323    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.545416 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545348    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.545416 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545359    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.545483 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545434    2565 policy_none.go:49] "None policy: Start"
Jul 06 19:48:22.545483 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545450    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 19:48:22.545483 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.545462    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 19:48:22.546388 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.546375    2565 policy_none.go:47] "Start"
Jul 06 19:48:22.584544 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.584530    2565 manager.go:337] "Starting Device Plugin manager"
Jul 06 19:48:22.584638 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.584560    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 19:48:22.584638 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.584569    2565 server.go:87] "Starting device plugin registration server"
Jul 06 19:48:22.584826 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.584813    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 19:48:22.584892 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.584829    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 19:48:22.584955 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.584940    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 19:48:22.585050 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.585026    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 19:48:22.585050 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.585041    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 19:48:22.585525 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.585507    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 19:48:22.585622 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.585549    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.647603 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.647574    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 19:48:22.648752 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.648736    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 19:48:22.648803 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.648775    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 19:48:22.648803 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.648795    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 19:48:22.648876 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.648827    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 19:48:22.651553 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.651536    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:22.685977 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.685946    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.686749 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.686728    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.686829 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.686761    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.686829 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.686772    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.686829 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.686796    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.695653 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.695612    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.695653 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.695634    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-142.ec2.internal\": node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.711515 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.711492    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.748901 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.748877    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"]
Jul 06 19:48:22.749011 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.748970    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.750391 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.750373    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.750496 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.750403    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.750496 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.750413    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.751575 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.751562    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.751703 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.751676    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.751740 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.751717    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.752293 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752278    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.752355 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752293    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.752355 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752307    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.752355 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752318    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.752355 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752321    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.752355 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.752330    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.753437 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.753419    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.753537 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.753451    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:48:22.754194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.754177    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:48:22.754292 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.754208    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:48:22.754292 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.754219    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:48:22.778920 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.778892    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-142.ec2.internal\" not found" node="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.783723 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.783708    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-142.ec2.internal\" not found" node="ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.811635 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.811614    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.826033 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.826017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/195224f1370a48f9ab6e557f0d094f7d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-142.ec2.internal\" (UID: \"195224f1370a48f9ab6e557f0d094f7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.826094 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.826079    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.826134 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.826106    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.915155 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:22.912750    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:22.927013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.926990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/195224f1370a48f9ab6e557f0d094f7d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-142.ec2.internal\" (UID: \"195224f1370a48f9ab6e557f0d094f7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.927075 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.927020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.927075 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.927042    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.927150 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.927083    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.927150 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.927110    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/195224f1370a48f9ab6e557f0d094f7d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-142.ec2.internal\" (UID: \"195224f1370a48f9ab6e557f0d094f7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:22.927150 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:22.927122    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc8ffd6820fbd2cdb2199c3a2b2c9d90-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal\" (UID: \"dc8ffd6820fbd2cdb2199c3a2b2c9d90\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:23.013450 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:23.013374    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:23.085313 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.085285    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:23.086622 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.086605    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:23.114503 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:23.114472    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:23.220866 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:23.220816    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:23.321571 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:23.321493    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:23.422624 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:23.422594    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-142.ec2.internal\" not found"
Jul 06 19:48:23.459873 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.459848    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 19:48:23.460364 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.460029    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 19:48:23.460364 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.460066    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 19:48:23.512224 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.512199    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:23.516721 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.516706    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:23.523030 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.523005    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 19:48:23.523030 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.523029    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:23.530274 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.530095    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:43:23 +0000 UTC" deadline="2028-02-10 13:50:34.921010265 +0000 UTC"
Jul 06 19:48:23.530274 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.530274    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14010h2m11.39074001s"
Jul 06 19:48:23.534921 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.534892    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 19:48:23.536016 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.535999    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal"
Jul 06 19:48:23.541511 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.541490    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 19:48:23.545173 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.545155    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 19:48:23.558251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.558232    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nzkk5"
Jul 06 19:48:23.565884 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.565860    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nzkk5"
Jul 06 19:48:23.645484 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:23.645450    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195224f1370a48f9ab6e557f0d094f7d.slice/crio-821f897194b56c3d0db293ee59cd17ff39777eecefca3688f6f6dbc5c6887609 WatchSource:0}: Error finding container 821f897194b56c3d0db293ee59cd17ff39777eecefca3688f6f6dbc5c6887609: Status 404 returned error can't find the container with id 821f897194b56c3d0db293ee59cd17ff39777eecefca3688f6f6dbc5c6887609
Jul 06 19:48:23.650511 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.650493    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:48:23.651840 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:23.651796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal" event={"ID":"195224f1370a48f9ab6e557f0d094f7d","Type":"ContainerStarted","Data":"821f897194b56c3d0db293ee59cd17ff39777eecefca3688f6f6dbc5c6887609"}
Jul 06 19:48:23.683049 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:23.683020    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8ffd6820fbd2cdb2199c3a2b2c9d90.slice/crio-7f04e936806bda2b1d042d8ac83aded38ea33202d7d3af861543d57a0f5bfaad WatchSource:0}: Error finding container 7f04e936806bda2b1d042d8ac83aded38ea33202d7d3af861543d57a0f5bfaad: Status 404 returned error can't find the container with id 7f04e936806bda2b1d042d8ac83aded38ea33202d7d3af861543d57a0f5bfaad
Jul 06 19:48:24.501190 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.501129    2565 apiserver.go:52] "Watching apiserver"
Jul 06 19:48:24.511307 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.511074    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 19:48:24.511613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.511584    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal","openshift-dns/node-resolver-42sf8","openshift-image-registry/node-ca-922hx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal","openshift-multus/multus-n9nkg","openshift-multus/network-metrics-daemon-pzghn","openshift-network-diagnostics/network-check-target-bth7t","openshift-network-operator/iptables-alerter-lhch9","kube-system/global-pull-secret-syncer-sh2cx","kube-system/konnectivity-agent-qsd29","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr","openshift-cluster-node-tuning-operator/tuned-v47dz","openshift-multus/multus-additional-cni-plugins-zkn8j","openshift-ovn-kubernetes/ovnkube-node-58fr7"]
Jul 06 19:48:24.513833 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.513178    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.515311 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.514588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.516051 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.515893    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qphsx\""
Jul 06 19:48:24.516169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.516079    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.516169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.516087    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.516169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.516128    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.516918 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.516893    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.517241 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.517223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g72mg\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.517443    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.517693    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.518321    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.518776    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519031    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519257    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519441    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zkwvf\""
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519758    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.519839    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519859    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.519904    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:24.520593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.519966    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.521341 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.521092    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.521341 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.521157    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:24.522838 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.522603    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.522838 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.522631    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.522838 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.522665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 19:48:24.522838 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.522835    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.523150 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.522950    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xttvv\""
Jul 06 19:48:24.524256 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.523793    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.525696 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.525317    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.525696 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.525480    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 19:48:24.526462 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526199    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 19:48:24.526462 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526428    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mt64b\""
Jul 06 19:48:24.527057 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526699    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.527057 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526779    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.527057 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526698    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 19:48:24.527057 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.526945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ljzt4\""
Jul 06 19:48:24.528494 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.527686    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.532205 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.531483    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.532205 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.531589    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8wh47\""
Jul 06 19:48:24.532205 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.532067    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.534183 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.534160    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.535549 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.534608    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 19:48:24.535549 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535107    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m6nzx\""
Jul 06 19:48:24.535549 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535113    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 19:48:24.535549 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535504    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-run\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-socket-dir-parent\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-var-lib-kubelet\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-bin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-lib-modules\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg64w\" (UniqueName: \"kubernetes.io/projected/dc51274b-629e-42ac-a025-e9ffafd363f5-kube-api-access-zg64w\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535775    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fa34b-0f62-40a7-86f7-c36abf2cdccb-konnectivity-ca\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a928635-994c-416e-aba8-35d28625546d-hosts-file\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.535874 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535836    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-kubelet\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.536491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.535895    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-hostroot\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.537299 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46zfk\" (UniqueName: \"kubernetes.io/projected/94ef51cc-2713-49f4-bc37-d199538fa6cc-kube-api-access-46zfk\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.537299 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-dbus\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.537867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-sys\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.537867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-host\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.537867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-cnibin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.537867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-registration-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.537867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537863    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-host-slash\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537935    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysconfig\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537951    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5jbvp\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537979    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-conf-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.537989    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538007    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-cni-binary-copy\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538017    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-kubernetes\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538057    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538070    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-sys-fs\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbxzf\" (UniqueName: \"kubernetes.io/projected/4a928635-994c-416e-aba8-35d28625546d-kube-api-access-pbxzf\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31a51b30-fae6-46a6-8202-a561d20de2e9-serviceca\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538175    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538198    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538220    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-socket-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.538265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-netns\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538311    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88ddz\" (UniqueName: \"kubernetes.io/projected/42f1c892-ac2d-473c-94b3-85a2778c481a-kube-api-access-88ddz\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538337    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-tmp\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538361    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31a51b30-fae6-46a6-8202-a561d20de2e9-host\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538420    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fa34b-0f62-40a7-86f7-c36abf2cdccb-agent-certs\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538437    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-device-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-etc-kubernetes\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538581    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-kubelet-config\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-systemd\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-daemon-config\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538775    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-iptables-alerter-script\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538803    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85pzs\" (UniqueName: \"kubernetes.io/projected/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-kube-api-access-85pzs\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538873    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-tuned\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538896    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb8tb\" (UniqueName: \"kubernetes.io/projected/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-kube-api-access-lb8tb\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.538982    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-os-release\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-k8s-cni-cncf-io\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-multus\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539148    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539223    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-modprobe-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-conf\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a928635-994c-416e-aba8-35d28625546d-tmp-dir\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqf9f\" (UniqueName: \"kubernetes.io/projected/31a51b30-fae6-46a6-8202-a561d20de2e9-kube-api-access-qqf9f\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-system-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.539759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.539491    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-multus-certs\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.568474 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.568404    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:43:23 +0000 UTC" deadline="2028-04-06 01:33:15.556965178 +0000 UTC"
Jul 06 19:48:24.568589 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.568476    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15341h44m50.988493707s"
Jul 06 19:48:24.625133 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.625034    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 19:48:24.640622 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640541    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640622 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640585    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-systemd\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640622 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-daemon-config\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.640622 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-var-lib-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640675    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-iptables-alerter-script\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85pzs\" (UniqueName: \"kubernetes.io/projected/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-kube-api-access-85pzs\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-tuned\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640796    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-systemd\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb8tb\" (UniqueName: \"kubernetes.io/projected/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-kube-api-access-lb8tb\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640882    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-os-release\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-k8s-cni-cncf-io\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.640984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.640980    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-k8s-cni-cncf-io\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641046    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-os-release\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641198    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-multus\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641242    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641331    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-modprobe-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641330    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-iptables-alerter-script\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641395    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-daemon-config\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641401    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641444    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.641450    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641449    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-multus\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-ovn\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641484    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-modprobe-d\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.641506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-log-socket\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.641526    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.141500383 +0000 UTC m=+3.403479239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-conf\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641599    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a928635-994c-416e-aba8-35d28625546d-tmp-dir\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqf9f\" (UniqueName: \"kubernetes.io/projected/31a51b30-fae6-46a6-8202-a561d20de2e9-kube-api-access-qqf9f\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-system-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-multus-certs\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641704    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-os-release\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641732    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641758    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-systemd-units\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-run\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-socket-dir-parent\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641731    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysctl-conf\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cnibin\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.641969    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:24.642194 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-var-lib-kubelet\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-bin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.641983    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-system-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.642051    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.142033632 +0000 UTC m=+3.404012498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642055    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a928635-994c-416e-aba8-35d28625546d-tmp-dir\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642070    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-config\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642078    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-cni-bin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642081    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-var-lib-kubelet\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptv6s\" (UniqueName: \"kubernetes.io/projected/3395b714-6786-482e-8ee4-75e554b601be-kube-api-access-ptv6s\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642128    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-lib-modules\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642169    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-socket-dir-parent\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642198    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-run\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642206    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-multus-certs\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642324    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg64w\" (UniqueName: \"kubernetes.io/projected/dc51274b-629e-42ac-a025-e9ffafd363f5-kube-api-access-zg64w\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fa34b-0f62-40a7-86f7-c36abf2cdccb-konnectivity-ca\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642383    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a928635-994c-416e-aba8-35d28625546d-hosts-file\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642384    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-lib-modules\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.642914 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-kubelet\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-hostroot\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46zfk\" (UniqueName: \"kubernetes.io/projected/94ef51cc-2713-49f4-bc37-d199538fa6cc-kube-api-access-46zfk\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-dbus\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642532    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-sys\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642535    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-var-lib-kubelet\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642557    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-host\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-cnibin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-registration-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-host-slash\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysconfig\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-conf-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642791    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-cni-binary-copy\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642810    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-hostroot\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-system-cni-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.643761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzfww\" (UniqueName: \"kubernetes.io/projected/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-kube-api-access-wzfww\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-kubernetes\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-sys-fs\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbxzf\" (UniqueName: \"kubernetes.io/projected/4a928635-994c-416e-aba8-35d28625546d-kube-api-access-pbxzf\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642974    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-dbus\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-netns\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-systemd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643046    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-sys\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643047    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-env-overrides\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643073    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a928635-994c-416e-aba8-35d28625546d-hosts-file\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643088    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3395b714-6786-482e-8ee4-75e554b601be-ovn-node-metrics-cert\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643119    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31a51b30-fae6-46a6-8202-a561d20de2e9-serviceca\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643124    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-registration-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643176    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-host-slash\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643243    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-cnibin\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643317    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-host\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.644678 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643372    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-kubernetes\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643419    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-conf-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643585    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31a51b30-fae6-46a6-8202-a561d20de2e9-serviceca\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.642952    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fa34b-0f62-40a7-86f7-c36abf2cdccb-konnectivity-ca\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643670    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-sys-fs\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643706    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-sysconfig\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643729    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-kubelet\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-etc-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-bin\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-socket-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-netns\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643845    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94ef51cc-2713-49f4-bc37-d199538fa6cc-cni-binary-copy\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643864    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-host-run-netns\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-slash\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-netd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.645512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643910    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88ddz\" (UniqueName: \"kubernetes.io/projected/42f1c892-ac2d-473c-94b3-85a2778c481a-kube-api-access-88ddz\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.643993    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-socket-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-tmp\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644028    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31a51b30-fae6-46a6-8202-a561d20de2e9-host\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644055    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-node-log\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fa34b-0f62-40a7-86f7-c36abf2cdccb-agent-certs\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644181    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-device-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644206    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-etc-kubernetes\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-kubelet-config\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-script-lib\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644397    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-multus-cni-dir\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644620    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31a51b30-fae6-46a6-8202-a561d20de2e9-host\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644671    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ef51cc-2713-49f4-bc37-d199538fa6cc-etc-kubernetes\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644720    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1605fa02-16bb-41b4-82fb-c934c9c21dc4-kubelet-config\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.644766    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f1c892-ac2d-473c-94b3-85a2778c481a-device-dir\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.646407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.646190    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-etc-tuned\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.647925 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.647891    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fa34b-0f62-40a7-86f7-c36abf2cdccb-agent-certs\") pod \"konnectivity-agent-qsd29\" (UID: \"653fa34b-0f62-40a7-86f7-c36abf2cdccb\") " pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.648581 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.648262    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-tmp\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.652726 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.652207    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:24.652726 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.652228    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:24.652726 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.652241    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:24.652726 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:24.652303    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.152286702 +0000 UTC m=+3.414265563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:24.658027 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.657948    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal" event={"ID":"dc8ffd6820fbd2cdb2199c3a2b2c9d90","Type":"ContainerStarted","Data":"7f04e936806bda2b1d042d8ac83aded38ea33202d7d3af861543d57a0f5bfaad"}
Jul 06 19:48:24.658776 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.658753    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88ddz\" (UniqueName: \"kubernetes.io/projected/42f1c892-ac2d-473c-94b3-85a2778c481a-kube-api-access-88ddz\") pod \"aws-ebs-csi-driver-node-p4ctr\" (UID: \"42f1c892-ac2d-473c-94b3-85a2778c481a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.659308 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.659260    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbxzf\" (UniqueName: \"kubernetes.io/projected/4a928635-994c-416e-aba8-35d28625546d-kube-api-access-pbxzf\") pod \"node-resolver-42sf8\" (UID: \"4a928635-994c-416e-aba8-35d28625546d\") " pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.660476 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.660437    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85pzs\" (UniqueName: \"kubernetes.io/projected/3bfb487b-3bca-4f53-905d-c65ca1e4aa97-kube-api-access-85pzs\") pod \"iptables-alerter-lhch9\" (UID: \"3bfb487b-3bca-4f53-905d-c65ca1e4aa97\") " pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.661589 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.661509    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46zfk\" (UniqueName: \"kubernetes.io/projected/94ef51cc-2713-49f4-bc37-d199538fa6cc-kube-api-access-46zfk\") pod \"multus-n9nkg\" (UID: \"94ef51cc-2713-49f4-bc37-d199538fa6cc\") " pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.662126 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.661715    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb8tb\" (UniqueName: \"kubernetes.io/projected/efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9-kube-api-access-lb8tb\") pod \"tuned-v47dz\" (UID: \"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9\") " pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.666469 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.662495    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqf9f\" (UniqueName: \"kubernetes.io/projected/31a51b30-fae6-46a6-8202-a561d20de2e9-kube-api-access-qqf9f\") pod \"node-ca-922hx\" (UID: \"31a51b30-fae6-46a6-8202-a561d20de2e9\") " pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.667351 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.666579    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg64w\" (UniqueName: \"kubernetes.io/projected/dc51274b-629e-42ac-a025-e9ffafd363f5-kube-api-access-zg64w\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:24.744892 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.744840    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-os-release\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.744892 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.744890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.744918    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-systemd-units\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.744980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cnibin\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745007    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745037    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-config\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745042    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-os-release\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745054    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-systemd-units\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745090    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cnibin\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745108    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745062    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptv6s\" (UniqueName: \"kubernetes.io/projected/3395b714-6786-482e-8ee4-75e554b601be-kube-api-access-ptv6s\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745210 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-system-cni-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzfww\" (UniqueName: \"kubernetes.io/projected/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-kube-api-access-wzfww\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-netns\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745287    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745324    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-system-cni-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745345    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-run-netns\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745405    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-systemd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-env-overrides\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745484    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-systemd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3395b714-6786-482e-8ee4-75e554b601be-ovn-node-metrics-cert\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745570    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.745746 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745633    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-config\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-kubelet\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745814    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-etc-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745840    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-bin\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745872    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745903    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-slash\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-netd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.745974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-node-log\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-script-lib\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-var-lib-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746058    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-env-overrides\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746065    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746109    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746141    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746155    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-kubelet\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-ovn\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746196    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-run-ovn\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.746461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746202    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-netd\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746200    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-slash\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-log-socket\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746248    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-var-lib-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746264    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-node-log\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746282    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-host-cni-bin\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746290    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-etc-openvswitch\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746340    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3395b714-6786-482e-8ee4-75e554b601be-log-socket\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.746762    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3395b714-6786-482e-8ee4-75e554b601be-ovnkube-script-lib\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.747453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.747301    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.748604 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.748558    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3395b714-6786-482e-8ee4-75e554b601be-ovn-node-metrics-cert\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.753625 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.753564    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:24.757604 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.757574    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzfww\" (UniqueName: \"kubernetes.io/projected/3b2a4ba1-2f43-45b8-9d5f-443c16c85870-kube-api-access-wzfww\") pod \"multus-additional-cni-plugins-zkn8j\" (UID: \"3b2a4ba1-2f43-45b8-9d5f-443c16c85870\") " pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.758054 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.758028    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptv6s\" (UniqueName: \"kubernetes.io/projected/3395b714-6786-482e-8ee4-75e554b601be-kube-api-access-ptv6s\") pod \"ovnkube-node-58fr7\" (UID: \"3395b714-6786-482e-8ee4-75e554b601be\") " pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.833048 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.833007    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42sf8"
Jul 06 19:48:24.854432 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.854405    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-922hx"
Jul 06 19:48:24.863323 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.863292    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9nkg"
Jul 06 19:48:24.874404 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.874380    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhch9"
Jul 06 19:48:24.882383 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.882335    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:24.891608 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.891511    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr"
Jul 06 19:48:24.899428 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.899406    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v47dz"
Jul 06 19:48:24.906251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.906170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zkn8j"
Jul 06 19:48:24.913064 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.913044    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:24.955422 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:24.955388    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:48:25.150360 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.150265    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:25.150360 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.150332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:25.150590 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.150395    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:25.150590 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.150449    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:25.150590 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.150516    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:26.150496685 +0000 UTC m=+4.412475540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:25.150590 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.150535    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:26.150525319 +0000 UTC m=+4.412504166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:25.251764 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.251722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:25.251979 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.251891    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:25.251979 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.251920    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:25.251979 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.251952    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:25.252155 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.252023    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:26.252003093 +0000 UTC m=+4.513981953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:25.569585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.569471    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:43:23 +0000 UTC" deadline="2028-01-02 12:57:19.963652858 +0000 UTC"
Jul 06 19:48:25.569585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.569525    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13073h8m54.394130492s"
Jul 06 19:48:25.649390 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:25.649352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:25.649576 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:25.649501    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:25.696747 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.696719    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe22e4e_1ac0_4bea_86d7_49e7a2ef1ae9.slice/crio-63d2cdf12958d000b94fcc86918da9f081196ed5cc63247aa240dc317f9a82e7 WatchSource:0}: Error finding container 63d2cdf12958d000b94fcc86918da9f081196ed5cc63247aa240dc317f9a82e7: Status 404 returned error can't find the container with id 63d2cdf12958d000b94fcc86918da9f081196ed5cc63247aa240dc317f9a82e7
Jul 06 19:48:25.714854 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.714828    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a51b30_fae6_46a6_8202_a561d20de2e9.slice/crio-a310c0ab0157e949f4449ccb41453aa53a52419b65ec70bb949e8442619259c6 WatchSource:0}: Error finding container a310c0ab0157e949f4449ccb41453aa53a52419b65ec70bb949e8442619259c6: Status 404 returned error can't find the container with id a310c0ab0157e949f4449ccb41453aa53a52419b65ec70bb949e8442619259c6
Jul 06 19:48:25.715970 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.715856    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a928635_994c_416e_aba8_35d28625546d.slice/crio-c02873d50f29f9d986a43f493e9a1153f8671b51024c7bf8f252b5e4e585212c WatchSource:0}: Error finding container c02873d50f29f9d986a43f493e9a1153f8671b51024c7bf8f252b5e4e585212c: Status 404 returned error can't find the container with id c02873d50f29f9d986a43f493e9a1153f8671b51024c7bf8f252b5e4e585212c
Jul 06 19:48:25.716837 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.716811    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3395b714_6786_482e_8ee4_75e554b601be.slice/crio-b3ca1eec01495642825987eb01819c2a769764638717578ff45778a7a446afb9 WatchSource:0}: Error finding container b3ca1eec01495642825987eb01819c2a769764638717578ff45778a7a446afb9: Status 404 returned error can't find the container with id b3ca1eec01495642825987eb01819c2a769764638717578ff45778a7a446afb9
Jul 06 19:48:25.717711 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.717620    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f1c892_ac2d_473c_94b3_85a2778c481a.slice/crio-cde08be23721606a77e587b41c0842d59245a133d9fef20909c994ba9a6f6a3c WatchSource:0}: Error finding container cde08be23721606a77e587b41c0842d59245a133d9fef20909c994ba9a6f6a3c: Status 404 returned error can't find the container with id cde08be23721606a77e587b41c0842d59245a133d9fef20909c994ba9a6f6a3c
Jul 06 19:48:25.718522 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.718498    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ef51cc_2713_49f4_bc37_d199538fa6cc.slice/crio-ad31199ca225a6b9bcf5867841044b9dabc98459fffb9cc4333b4faeb56b212e WatchSource:0}: Error finding container ad31199ca225a6b9bcf5867841044b9dabc98459fffb9cc4333b4faeb56b212e: Status 404 returned error can't find the container with id ad31199ca225a6b9bcf5867841044b9dabc98459fffb9cc4333b4faeb56b212e
Jul 06 19:48:25.720009 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.719939    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653fa34b_0f62_40a7_86f7_c36abf2cdccb.slice/crio-2b3d88dd0dfa7f9832b57c92e937cd9911553f346d6ed3659cce6440136aa504 WatchSource:0}: Error finding container 2b3d88dd0dfa7f9832b57c92e937cd9911553f346d6ed3659cce6440136aa504: Status 404 returned error can't find the container with id 2b3d88dd0dfa7f9832b57c92e937cd9911553f346d6ed3659cce6440136aa504
Jul 06 19:48:25.721891 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.721869    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfb487b_3bca_4f53_905d_c65ca1e4aa97.slice/crio-f1de837758e0e0848f245be2ae1f3b8028c6eea4636b96226597c2741ec5dfa7 WatchSource:0}: Error finding container f1de837758e0e0848f245be2ae1f3b8028c6eea4636b96226597c2741ec5dfa7: Status 404 returned error can't find the container with id f1de837758e0e0848f245be2ae1f3b8028c6eea4636b96226597c2741ec5dfa7
Jul 06 19:48:25.722779 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:25.722694    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2a4ba1_2f43_45b8_9d5f_443c16c85870.slice/crio-92b0c99c7360c58f99a271b58b02be9abbb3c31190d36f9c219e41c3d14b8a90 WatchSource:0}: Error finding container 92b0c99c7360c58f99a271b58b02be9abbb3c31190d36f9c219e41c3d14b8a90: Status 404 returned error can't find the container with id 92b0c99c7360c58f99a271b58b02be9abbb3c31190d36f9c219e41c3d14b8a90
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.159237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.159350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.159514    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.159590    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:28.159570484 +0000 UTC m=+6.421549349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.159606    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:26.159997 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.159689    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:28.159669316 +0000 UTC m=+6.421648173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:26.260572 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.260484    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:26.260752 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.260694    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:26.260752 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.260726    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:26.260752 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.260741    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:26.260913 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.260806    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:28.260785233 +0000 UTC m=+6.522764093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:26.650338 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.650072    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:26.651149 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.650459    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:26.651149 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.650107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:26.651297 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:26.651204    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:26.681684 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.681647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9nkg" event={"ID":"94ef51cc-2713-49f4-bc37-d199538fa6cc","Type":"ContainerStarted","Data":"ad31199ca225a6b9bcf5867841044b9dabc98459fffb9cc4333b4faeb56b212e"}
Jul 06 19:48:26.693333 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.693294    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerStarted","Data":"92b0c99c7360c58f99a271b58b02be9abbb3c31190d36f9c219e41c3d14b8a90"}
Jul 06 19:48:26.708684 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.708585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42sf8" event={"ID":"4a928635-994c-416e-aba8-35d28625546d","Type":"ContainerStarted","Data":"c02873d50f29f9d986a43f493e9a1153f8671b51024c7bf8f252b5e4e585212c"}
Jul 06 19:48:26.727954 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.727917    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal" event={"ID":"195224f1370a48f9ab6e557f0d094f7d","Type":"ContainerStarted","Data":"f69558de26e94f6fa53d9fdd74859ed4cb58fa2e4daea0880fdbee4919d1cb58"}
Jul 06 19:48:26.743283 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.743243    2565 generic.go:356] "Generic (PLEG): container finished" podID="dc8ffd6820fbd2cdb2199c3a2b2c9d90" containerID="b904db306f7888127b60a70a3bcdcb165d9c475bd00f2deee864b37ab8d90c27" exitCode=0
Jul 06 19:48:26.743454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.743335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal" event={"ID":"dc8ffd6820fbd2cdb2199c3a2b2c9d90","Type":"ContainerDied","Data":"b904db306f7888127b60a70a3bcdcb165d9c475bd00f2deee864b37ab8d90c27"}
Jul 06 19:48:26.766119 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.766062    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-142.ec2.internal" podStartSLOduration=3.766039483 podStartE2EDuration="3.766039483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:26.747810659 +0000 UTC m=+5.009789527" watchObservedRunningTime="2026-07-06 19:48:26.766039483 +0000 UTC m=+5.028018353"
Jul 06 19:48:26.770274 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.770246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhch9" event={"ID":"3bfb487b-3bca-4f53-905d-c65ca1e4aa97","Type":"ContainerStarted","Data":"f1de837758e0e0848f245be2ae1f3b8028c6eea4636b96226597c2741ec5dfa7"}
Jul 06 19:48:26.787158 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.787060    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr" event={"ID":"42f1c892-ac2d-473c-94b3-85a2778c481a","Type":"ContainerStarted","Data":"cde08be23721606a77e587b41c0842d59245a133d9fef20909c994ba9a6f6a3c"}
Jul 06 19:48:26.791736 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.791705    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-922hx" event={"ID":"31a51b30-fae6-46a6-8202-a561d20de2e9","Type":"ContainerStarted","Data":"a310c0ab0157e949f4449ccb41453aa53a52419b65ec70bb949e8442619259c6"}
Jul 06 19:48:26.802817 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.802777    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v47dz" event={"ID":"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9","Type":"ContainerStarted","Data":"63d2cdf12958d000b94fcc86918da9f081196ed5cc63247aa240dc317f9a82e7"}
Jul 06 19:48:26.808148 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.808118    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"b3ca1eec01495642825987eb01819c2a769764638717578ff45778a7a446afb9"}
Jul 06 19:48:26.827083 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:26.827052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qsd29" event={"ID":"653fa34b-0f62-40a7-86f7-c36abf2cdccb","Type":"ContainerStarted","Data":"2b3d88dd0dfa7f9832b57c92e937cd9911553f346d6ed3659cce6440136aa504"}
Jul 06 19:48:27.649226 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:27.649190    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:27.649411 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:27.649332    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:27.845160 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:27.845055    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal" event={"ID":"dc8ffd6820fbd2cdb2199c3a2b2c9d90","Type":"ContainerStarted","Data":"bb0511ed9295978b467971ef26f7972877365d207fef1ccb2f55351d0599f281"}
Jul 06 19:48:28.176891 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:28.176851    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:28.177095 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:28.176910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:28.177095 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.177044    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:28.177207 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.177106    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:32.17708512 +0000 UTC m=+10.439063968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:28.177540 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.177519    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:28.177619 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.177580    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:32.17756378 +0000 UTC m=+10.439542625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:28.278302 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:28.278268    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:28.278529 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.278478    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:28.278529 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.278507    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:28.278529 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.278522    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:28.278706 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.278603    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:32.278580112 +0000 UTC m=+10.540559186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:28.650197 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:28.650111    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:28.650409 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.650268    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:28.650868 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:28.650845    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:28.651010 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:28.650986    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:29.649888 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:29.649858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:29.650374 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:29.649993    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:30.650059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:30.650025    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:30.650059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:30.650057    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:30.650594 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:30.650162    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:30.650594 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:30.650269    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:31.649131 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:31.649074    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:31.649317 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:31.649198    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:32.212959 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:32.212830    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:32.212959 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:32.212886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:32.213474 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.213002    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:32.213474 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.213079    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:40.213058914 +0000 UTC m=+18.475037770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:32.213474 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.213013    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:32.213474 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.213143    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:40.213124788 +0000 UTC m=+18.475103646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:32.313500 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:32.313462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:32.313671 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.313636    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:32.313671 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.313662    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:32.313795 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.313678    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:32.313795 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.313736    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:40.313717399 +0000 UTC m=+18.575696252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:32.661216 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:32.660904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:32.661216 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.661044    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:32.661993 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:32.661478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:32.661993 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:32.661615    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:33.649926 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:33.649881    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:33.650401 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:33.650030    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:34.650134 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:34.650096    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:34.650598 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:34.650241    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:34.650660 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:34.650634    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:34.650759 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:34.650738    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:35.649967 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:35.649938    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:35.650134 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:35.650040    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:36.649169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:36.649142    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:36.649169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:36.649160    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:36.649559 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:36.649244    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:36.649559 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:36.649314    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:37.649646 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:37.649611    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:37.650129 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:37.649736    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:38.649802 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:38.649764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:38.649802 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:38.649798    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:38.650349 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:38.649896    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:38.650349 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:38.650020    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:39.649125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:39.649089    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:39.649373 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:39.649195    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:40.272085 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:40.272049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:40.272510 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:40.272101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:40.272510 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.272211    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:40.272510 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.272252    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:40.272510 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.272279    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:56.272264801 +0000 UTC m=+34.534243643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:40.272510 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.272304    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:56.272286336 +0000 UTC m=+34.534265183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:40.373193 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:40.373158    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:40.373347 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.373326    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:40.373347 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.373348    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:40.373439 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.373361    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:40.373439 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.373423    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:56.373404861 +0000 UTC m=+34.635383707 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:40.650034 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:40.649955    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:40.650034 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:40.650003    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:40.650228 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.650086    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:40.650228 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:40.650210    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:41.649145 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:41.649103    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:41.649584 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:41.649213    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:42.650601 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:42.650572    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:42.651211 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:42.650665    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:42.651344 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:42.651247    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:42.651344 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:42.651304    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:43.649971 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:43.649939    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:43.650133 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:43.650048    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:44.650961 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.649227    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:44.650961 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:44.649442    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:44.650961 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.649228    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:44.651378 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:44.650976    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:44.880757 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.880532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr" event={"ID":"42f1c892-ac2d-473c-94b3-85a2778c481a","Type":"ContainerStarted","Data":"02c28c13ee9fc190e124af915cf2d349c04f7e28e6f4e1f17dc849779e044028"}
Jul 06 19:48:44.881925 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.881891    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-922hx" event={"ID":"31a51b30-fae6-46a6-8202-a561d20de2e9","Type":"ContainerStarted","Data":"229eb808245ff26ee791db9af4ddddf4e0f380f6d89614b037b804e991742fb7"}
Jul 06 19:48:44.883603 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.883580    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v47dz" event={"ID":"efe22e4e-1ac0-4bea-86d7-49e7a2ef1ae9","Type":"ContainerStarted","Data":"361a57f85291687056ec90faf8074df75caea2533a803ac43184466df04d6f1a"}
Jul 06 19:48:44.885291 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.885268    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"5e590668ee1fe4a1b3fc3aa990ff48670a9a16ad06f2cb7100aff57b1d00bfea"}
Jul 06 19:48:44.885372 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.885340    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"40a9ddedfdfe47b856f709d3bda29a4b5d93c100b8ab0c1c808a1597fe4f3c43"}
Jul 06 19:48:44.886503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.886478    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qsd29" event={"ID":"653fa34b-0f62-40a7-86f7-c36abf2cdccb","Type":"ContainerStarted","Data":"485d21d1197e6ce4b892460ce3213450c0b7e9000bc0707a6431900e42888373"}
Jul 06 19:48:44.887876 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.887854    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9nkg" event={"ID":"94ef51cc-2713-49f4-bc37-d199538fa6cc","Type":"ContainerStarted","Data":"e20464adbf39357366f8d35cd2c728b3cba6672c7f06f517bd5bc238164c42d4"}
Jul 06 19:48:44.889190 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.889168    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerStarted","Data":"71e1698e70e01019214e1a2d3d0b38ca1de7386d841abb93f2793f60769a313f"}
Jul 06 19:48:44.890453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.890435    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42sf8" event={"ID":"4a928635-994c-416e-aba8-35d28625546d","Type":"ContainerStarted","Data":"c394a575b0b50af0f0e97827d1785b597daeb1502cac0456e0c763504a3d0f60"}
Jul 06 19:48:44.896327 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.896290    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-142.ec2.internal" podStartSLOduration=21.896279757 podStartE2EDuration="21.896279757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:27.863689965 +0000 UTC m=+6.125668835" watchObservedRunningTime="2026-07-06 19:48:44.896279757 +0000 UTC m=+23.158258621"
Jul 06 19:48:44.910022 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.909988    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qsd29" podStartSLOduration=6.789371111 podStartE2EDuration="21.909971644s" totalImagesPullingTime="15.120600533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.909817336 +0000 UTC m=+23.171796202" watchObservedRunningTime="2026-07-06 19:48:44.909971644 +0000 UTC m=+23.171950508"
Jul 06 19:48:44.910387 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.910367    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-922hx" podStartSLOduration=3.131399291 podStartE2EDuration="21.910361659s" totalImagesPullingTime="18.778962368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.896270259 +0000 UTC m=+23.158249122" watchObservedRunningTime="2026-07-06 19:48:44.910361659 +0000 UTC m=+23.172340523"
Jul 06 19:48:44.925578 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.925545    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v47dz" podStartSLOduration=3.104984761 podStartE2EDuration="21.925532132s" totalImagesPullingTime="18.820547371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.92531566 +0000 UTC m=+23.187294542" watchObservedRunningTime="2026-07-06 19:48:44.925532132 +0000 UTC m=+23.187510996"
Jul 06 19:48:44.962868 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:44.962813    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-42sf8" podStartSLOduration=3.180792181 podStartE2EDuration="21.962790571s" totalImagesPullingTime="18.78199839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.962074006 +0000 UTC m=+23.224052871" watchObservedRunningTime="2026-07-06 19:48:44.962790571 +0000 UTC m=+23.224769528"
Jul 06 19:48:45.649815 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.649560    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:45.650005 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:45.649924    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:45.893222 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.893146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhch9" event={"ID":"3bfb487b-3bca-4f53-905d-c65ca1e4aa97","Type":"ContainerStarted","Data":"999ad784ed4d5426bd36c33685063cf34a5b8d9fb3e7de75f797b914289c1020"}
Jul 06 19:48:45.895635 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.895615    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:48:45.896013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.895993    2565 generic.go:356] "Generic (PLEG): container finished" podID="3395b714-6786-482e-8ee4-75e554b601be" containerID="5e590668ee1fe4a1b3fc3aa990ff48670a9a16ad06f2cb7100aff57b1d00bfea" exitCode=1
Jul 06 19:48:45.896084 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.896067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerDied","Data":"5e590668ee1fe4a1b3fc3aa990ff48670a9a16ad06f2cb7100aff57b1d00bfea"}
Jul 06 19:48:45.896164 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.896148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"33ad3856d350bad03feddf025109351044444bcc14c2020c645276c20875ebe5"}
Jul 06 19:48:45.896213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.896200    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"78427b997502281f5e3c5dfc4fea102233fbf6192e9b37bac6611e241820a74d"}
Jul 06 19:48:45.896264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.896231    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"3f61f3eb5a84c96a60150f1b9699be83857643bdb45f1267ff886cbf2ad328e0"}
Jul 06 19:48:45.896301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.896278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"6582f19da0533dc9cc89dbadd57a0c968e99d864b3cffa9d6e03c161891dd7f1"}
Jul 06 19:48:45.897399 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.897368    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="71e1698e70e01019214e1a2d3d0b38ca1de7386d841abb93f2793f60769a313f" exitCode=0
Jul 06 19:48:45.897478 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.897441    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"71e1698e70e01019214e1a2d3d0b38ca1de7386d841abb93f2793f60769a313f"}
Jul 06 19:48:45.907299 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.907266    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n9nkg" podStartSLOduration=4.074533398 podStartE2EDuration="22.907253339s" totalImagesPullingTime="18.832719941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.978491368 +0000 UTC m=+23.240470234" watchObservedRunningTime="2026-07-06 19:48:45.907253339 +0000 UTC m=+24.169232197"
Jul 06 19:48:45.907498 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:45.907477    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lhch9" podStartSLOduration=4.137854303 podStartE2EDuration="22.907470078s" totalImagesPullingTime="18.769615775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:45.907239411 +0000 UTC m=+24.169218276" watchObservedRunningTime="2026-07-06 19:48:45.907470078 +0000 UTC m=+24.169448941"
Jul 06 19:48:46.252354 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.252332    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 19:48:46.596118 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.595951    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:48:46.252349562Z","UUID":"881068e5-f44e-4447-977f-d00e6c66da29","Handler":null,"Name":"","Endpoint":""}
Jul 06 19:48:46.598860 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.598837    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 19:48:46.598997 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.598874    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 19:48:46.649876 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.649749    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:46.650045 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:46.649878    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:46.650045 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.649929    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:46.650045 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:46.650019    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:46.902091 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:46.902017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr" event={"ID":"42f1c892-ac2d-473c-94b3-85a2778c481a","Type":"ContainerStarted","Data":"3a8cb73f138fbdcd78f415d0da23bbcb9fab0ece70fb77cce26a39d5bc552d8d"}
Jul 06 19:48:47.649151 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.649076    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:47.649339 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:47.649196    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:47.906189 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.906093    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr" event={"ID":"42f1c892-ac2d-473c-94b3-85a2778c481a","Type":"ContainerStarted","Data":"06169581d79908ff885fc5c03364da51405e1a3d17f10788995927bf80df994f"}
Jul 06 19:48:47.908770 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.908750    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:48:47.909098 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.909079    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"db0bf31b5fc3c72cc02771e55be2fc94f589ea408ffd428a907b98a1b272ff67"}
Jul 06 19:48:47.920880 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.920864    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:47.921313 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.921298    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:47.925503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:47.925470    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4ctr" podStartSLOduration=3.842435258 podStartE2EDuration="24.925458729s" totalImagesPullingTime="21.083023471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:47.925277257 +0000 UTC m=+26.187256132" watchObservedRunningTime="2026-07-06 19:48:47.925458729 +0000 UTC m=+26.187437608"
Jul 06 19:48:48.649477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:48.649426    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:48.649477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:48.649478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:48.649677 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:48.649593    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:48.649723 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:48.649707    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:48.911542 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:48.911459    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:48.912412 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:48.912391    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qsd29"
Jul 06 19:48:49.649504 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:49.649333    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:49.649655 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:49.649591    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:50.649134 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.649062    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:50.649134 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.649085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:50.649833 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:50.649159    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:50.649833 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:50.649300    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:50.917441 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.917254    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:48:50.917736 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.917718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"e4aa48088a667511c62014332b912fc9965eb83ec67c121c252fdd83a25a7fe5"}
Jul 06 19:48:50.918050 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.917901    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:50.918050 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.917975    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:50.918050 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.918022    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:50.918241 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.918122    2565 scope.go:117] "RemoveContainer" containerID="5e590668ee1fe4a1b3fc3aa990ff48670a9a16ad06f2cb7100aff57b1d00bfea"
Jul 06 19:48:50.919737 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.919712    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="0190ca15868d2c3dd4ee1574cec90ed6113ad7b4f2cbbf7203fd2e9ae3c5b135" exitCode=0
Jul 06 19:48:50.919924 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.919806    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"0190ca15868d2c3dd4ee1574cec90ed6113ad7b4f2cbbf7203fd2e9ae3c5b135"}
Jul 06 19:48:50.933925 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.933891    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:50.934080 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:50.934067    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:48:51.650039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.650002    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:51.650605 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:51.650103    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:51.925206 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.925160    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:48:51.925547 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.925519    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" event={"ID":"3395b714-6786-482e-8ee4-75e554b601be","Type":"ContainerStarted","Data":"9190092335d94f2aed9bf36ae09ecf2e8ebb8f1e3ef9cb5e21988e2e89c192e4"}
Jul 06 19:48:51.927452 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.927427    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="1d46c70c7e0fc316a25e90d23ed9f6b2721dc83a4fb4865dd78faf516f6179e1" exitCode=0
Jul 06 19:48:51.927543 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.927464    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"1d46c70c7e0fc316a25e90d23ed9f6b2721dc83a4fb4865dd78faf516f6179e1"}
Jul 06 19:48:51.945793 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.945773    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sh2cx"]
Jul 06 19:48:51.945872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.945858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:51.946000 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:51.945961    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:51.949136 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.949097    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bth7t"]
Jul 06 19:48:51.949238 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.949172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:51.949305 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:51.949261    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:51.949713 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.949696    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pzghn"]
Jul 06 19:48:51.949785 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.949774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:51.949871 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:51.949851    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:51.957469 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:51.957431    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7" podStartSLOduration=10.096250773 podStartE2EDuration="28.957419341s" totalImagesPullingTime="18.861168568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:51.957072197 +0000 UTC m=+30.219051062" watchObservedRunningTime="2026-07-06 19:48:51.957419341 +0000 UTC m=+30.219398216"
Jul 06 19:48:53.649161 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:53.648984    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:53.649543 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:53.649038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:53.649543 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:53.649231    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:53.649543 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:53.649038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:53.649543 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:53.649301    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:53.649543 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:53.649388    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:53.934692 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:53.934622    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="f868d99847159c4a69ae56e13a9cc635984c0a04cc63c18b3528d3f5aaa8c853" exitCode=0
Jul 06 19:48:53.934692 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:53.934667    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"f868d99847159c4a69ae56e13a9cc635984c0a04cc63c18b3528d3f5aaa8c853"}
Jul 06 19:48:55.057679 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:55.057644    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qsd29_653fa34b-0f62-40a7-86f7-c36abf2cdccb/konnectivity-agent/0.log" limit=1024
Jul 06 19:48:55.156349 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:55.156316    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qsd29_653fa34b-0f62-40a7-86f7-c36abf2cdccb/konnectivity-agent/0.log" limit=1024
Jul 06 19:48:55.649182 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:55.649150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:55.649182 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:55.649174    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:55.649421 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:55.649165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:55.649421 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:55.649277    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzghn" podUID="dc51274b-629e-42ac-a025-e9ffafd363f5"
Jul 06 19:48:55.649421 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:55.649349    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bth7t" podUID="a6290485-982b-4dd8-be92-f44b13ab3979"
Jul 06 19:48:55.649543 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:55.649430    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh2cx" podUID="1605fa02-16bb-41b4-82fb-c934c9c21dc4"
Jul 06 19:48:56.289457 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:56.289370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:56.289457 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:56.289440    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:56.289876 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.289537    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:56.289876 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.289537    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:56.289876 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.289602    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret podName:1605fa02-16bb-41b4-82fb-c934c9c21dc4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.289584538 +0000 UTC m=+66.551563383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret") pod "global-pull-secret-syncer-sh2cx" (UID: "1605fa02-16bb-41b4-82fb-c934c9c21dc4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:48:56.289876 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.289622    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.289612457 +0000 UTC m=+66.551591304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:48:56.390697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:56.390663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:56.390853 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.390802    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:48:56.390853 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.390816    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:48:56.390853 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.390826    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mc2dx for pod openshift-network-diagnostics/network-check-target-bth7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:56.390997 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:56.390869    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx podName:a6290485-982b-4dd8-be92-f44b13ab3979 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.390857978 +0000 UTC m=+66.652836820 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc2dx" (UniqueName: "kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx") pod "network-check-target-bth7t" (UID: "a6290485-982b-4dd8-be92-f44b13ab3979") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:48:57.543284 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.543216    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-142.ec2.internal" event="NodeReady"
Jul 06 19:48:57.543698 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.543325    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 19:48:57.587950 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.587924    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-24ntk"]
Jul 06 19:48:57.589899 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.589883    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.590830 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.590814    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tsgnw"]
Jul 06 19:48:57.592256 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.592229    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qb4h6\""
Jul 06 19:48:57.592256 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.592244    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 19:48:57.592378 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.592290    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 19:48:57.592421 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.592411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.594654 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.594635    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 19:48:57.594741 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.594635    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 19:48:57.595317 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.595303    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 19:48:57.595401 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.595383    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mdxh6\""
Jul 06 19:48:57.602899 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.602866    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-24ntk"]
Jul 06 19:48:57.608605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.608587    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tsgnw"]
Jul 06 19:48:57.649292 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.649273    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:48:57.649392 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.649274    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:48:57.649458 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.649276    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:48:57.653604 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.653585    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 19:48:57.654039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.653879    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 19:48:57.654039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.653891    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 19:48:57.654039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.653933    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xsp7\""
Jul 06 19:48:57.654039 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.653954    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8fm7\""
Jul 06 19:48:57.654264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.654112    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 19:48:57.700483 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700463    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52abd723-4417-452c-b72a-b3f5800f2762-config-volume\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.700570 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700489    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.700570 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhxx\" (UniqueName: \"kubernetes.io/projected/9731f41f-d036-4b90-a4ec-b6963f26c4cb-kube-api-access-8rhxx\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.700641 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700587    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9731f41f-d036-4b90-a4ec-b6963f26c4cb-cert\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.700641 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52abd723-4417-452c-b72a-b3f5800f2762-tmp-dir\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.700703 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.700662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrckc\" (UniqueName: \"kubernetes.io/projected/52abd723-4417-452c-b72a-b3f5800f2762-kube-api-access-wrckc\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.801836 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.801765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52abd723-4417-452c-b72a-b3f5800f2762-config-volume\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.801836 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.801801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.801859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhxx\" (UniqueName: \"kubernetes.io/projected/9731f41f-d036-4b90-a4ec-b6963f26c4cb-kube-api-access-8rhxx\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:57.801885    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.801935    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9731f41f-d036-4b90-a4ec-b6963f26c4cb-cert\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:57.801955    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:58.301935652 +0000 UTC m=+36.563914508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.801979    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52abd723-4417-452c-b72a-b3f5800f2762-tmp-dir\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.802059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.802032    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrckc\" (UniqueName: \"kubernetes.io/projected/52abd723-4417-452c-b72a-b3f5800f2762-kube-api-access-wrckc\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.802605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.802582    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52abd723-4417-452c-b72a-b3f5800f2762-tmp-dir\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.802795 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.802775    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52abd723-4417-452c-b72a-b3f5800f2762-config-volume\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.806354 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.806336    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9731f41f-d036-4b90-a4ec-b6963f26c4cb-cert\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.812994 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.812936    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhxx\" (UniqueName: \"kubernetes.io/projected/9731f41f-d036-4b90-a4ec-b6963f26c4cb-kube-api-access-8rhxx\") pod \"ingress-canary-tsgnw\" (UID: \"9731f41f-d036-4b90-a4ec-b6963f26c4cb\") " pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:57.814204 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.813491    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrckc\" (UniqueName: \"kubernetes.io/projected/52abd723-4417-452c-b72a-b3f5800f2762-kube-api-access-wrckc\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:57.904945 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:57.904901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tsgnw"
Jul 06 19:48:58.063625 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:58.063379    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tsgnw"]
Jul 06 19:48:58.305665 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:58.305630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:58.305856 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:58.305762    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:48:58.305856 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:58.305821    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:59.305806876 +0000 UTC m=+37.567785722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:48:59.314676 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:59.314642    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:48:59.315364 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:59.314775    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:48:59.315364 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:48:59.314834    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:01.314819059 +0000 UTC m=+39.576797901 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:48:59.449976 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:48:59.449943    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9731f41f_d036_4b90_a4ec_b6963f26c4cb.slice/crio-df0e36e5f8b5c66e61355da430a0a834357468083f84412aec5ff2efb57572d6 WatchSource:0}: Error finding container df0e36e5f8b5c66e61355da430a0a834357468083f84412aec5ff2efb57572d6: Status 404 returned error can't find the container with id df0e36e5f8b5c66e61355da430a0a834357468083f84412aec5ff2efb57572d6
Jul 06 19:48:59.951290 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:59.951219    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="126ac7bc2761b25aac9fe132132e653dd8a51ff65973695b80829498834f041a" exitCode=0
Jul 06 19:48:59.951439 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:59.951305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"126ac7bc2761b25aac9fe132132e653dd8a51ff65973695b80829498834f041a"}
Jul 06 19:48:59.952331 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:48:59.952311    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tsgnw" event={"ID":"9731f41f-d036-4b90-a4ec-b6963f26c4cb","Type":"ContainerStarted","Data":"df0e36e5f8b5c66e61355da430a0a834357468083f84412aec5ff2efb57572d6"}
Jul 06 19:49:00.957313 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:00.957285    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b2a4ba1-2f43-45b8-9d5f-443c16c85870" containerID="6d6bdfe3d312b2effe87f05b2a500d169ed8ac8234610213bf6a5fee4fe2cb07" exitCode=0
Jul 06 19:49:00.957701 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:00.957326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerDied","Data":"6d6bdfe3d312b2effe87f05b2a500d169ed8ac8234610213bf6a5fee4fe2cb07"}
Jul 06 19:49:01.333108 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:01.333031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:49:01.333265 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:01.333155    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:49:01.333265 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:01.333214    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:05.333196079 +0000 UTC m=+43.595174921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:49:01.964427 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:01.964386    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" event={"ID":"3b2a4ba1-2f43-45b8-9d5f-443c16c85870","Type":"ContainerStarted","Data":"bb292b4a47230d7f9a53663028e114e6839167a90020f9c83539340b849dddae"}
Jul 06 19:49:02.968368 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:02.968278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tsgnw" event={"ID":"9731f41f-d036-4b90-a4ec-b6963f26c4cb","Type":"ContainerStarted","Data":"7f1090efab4c8a8c68fc9a52c7a9032d7740f749577c5fbbc77516ea74e7a663"}
Jul 06 19:49:02.987187 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:02.987149    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zkn8j" podStartSLOduration=9.452564454000001 podStartE2EDuration="39.987135624s" totalImagesPullingTime="30.53457117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:01.989951311 +0000 UTC m=+40.251930177" watchObservedRunningTime="2026-07-06 19:49:02.987135624 +0000 UTC m=+41.249114487"
Jul 06 19:49:02.987558 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:02.987537    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tsgnw" podStartSLOduration=3.279026703 podStartE2EDuration="5.987531783s" totalImagesPullingTime="2.70850508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:02.985398637 +0000 UTC m=+41.247377513" watchObservedRunningTime="2026-07-06 19:49:02.987531783 +0000 UTC m=+41.249510646"
Jul 06 19:49:05.362702 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:05.362662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:49:05.363092 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:05.362788    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:49:05.363092 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:05.362844    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:13.362828832 +0000 UTC m=+51.624807679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:49:13.411324 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:13.411291    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:49:13.411761 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:13.411388    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:49:13.411761 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:13.411439    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:29.411423978 +0000 UTC m=+67.673402822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:49:22.939790 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:22.939766    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58fr7"
Jul 06 19:49:24.829000 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.828967    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"]
Jul 06 19:49:24.860208 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.860183    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"]
Jul 06 19:49:24.860338 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.860272    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.862739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.862716    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 19:49:24.862869 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.862764    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:49:24.862869 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.862779    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 19:49:24.863836 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.863817    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 19:49:24.863836 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.863826    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wtkfb\""
Jul 06 19:49:24.885867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.885841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.885969 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.885879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5nqr\" (UniqueName: \"kubernetes.io/projected/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-kube-api-access-h5nqr\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.885969 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.885899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.928251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.928227    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"]
Jul 06 19:49:24.950174 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.950144    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"]
Jul 06 19:49:24.950312 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.950292    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:24.952814 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.952791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 06 19:49:24.952942 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.952835    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 06 19:49:24.953010 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.952943    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 06 19:49:24.953056 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.953028    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:49:24.953112 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.953091    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qwp88\""
Jul 06 19:49:24.968828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.968812    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"
Jul 06 19:49:24.968828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.968820    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"]
Jul 06 19:49:24.971431 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.971412    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9v46z\""
Jul 06 19:49:24.971834 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.971817    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 06 19:49:24.972183 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.972164    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:49:24.978667 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.978652    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"]
Jul 06 19:49:24.978775 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.978762    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:24.982660 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.982643    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 19:49:24.982793 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.982644    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nbhq2\""
Jul 06 19:49:24.982962 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.982900    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 19:49:24.983043 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.983006    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 19:49:24.983043 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.983018    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 19:49:24.986265 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.986346 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb51b3d-3b09-4b5c-9a98-224123e80b50-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:24.986415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986354    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:24.986415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986379    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjtgp\" (UniqueName: \"kubernetes.io/projected/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-kube-api-access-vjtgp\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:24.986494 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbqhh\" (UniqueName: \"kubernetes.io/projected/7c943477-a00f-49fd-b404-20b64c51702b-kube-api-access-lbqhh\") pod \"volume-data-source-validator-5d456fdbcd-jjvr6\" (UID: \"7c943477-a00f-49fd-b404-20b64c51702b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"
Jul 06 19:49:24.986531 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.986531 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986524    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb51b3d-3b09-4b5c-9a98-224123e80b50-config\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:24.986628 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txcr7\" (UniqueName: \"kubernetes.io/projected/fdb51b3d-3b09-4b5c-9a98-224123e80b50-kube-api-access-txcr7\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:24.986628 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:24.986725 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986638    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5nqr\" (UniqueName: \"kubernetes.io/projected/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-kube-api-access-h5nqr\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.986958 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.986941    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.989695 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.989681    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.994315 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.994296    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5nqr\" (UniqueName: \"kubernetes.io/projected/a34011c5-eb88-4ca9-9b3f-26b0235ccc09-kube-api-access-h5nqr\") pod \"kube-storage-version-migrator-operator-846bf668f7-hjc86\" (UID: \"a34011c5-eb88-4ca9-9b3f-26b0235ccc09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:24.996578 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.996565    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"]
Jul 06 19:49:24.996644 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.996588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"]
Jul 06 19:49:24.996644 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.996601    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"]
Jul 06 19:49:24.996644 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.996614    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"]
Jul 06 19:49:24.996790 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.996683    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:24.999383 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.999366    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 19:49:24.999383 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.999369    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fcvl7\""
Jul 06 19:49:24.999383 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:24.999370    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 19:49:25.031882 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.031860    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:49:25.052227 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.052209    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:49:25.052301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.052280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.054657 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.054636    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 19:49:25.054744 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.054640    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g9x2b\""
Jul 06 19:49:25.054807 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.054750    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 19:49:25.054807 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.054764    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 19:49:25.060674 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.060657    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 19:49:25.087232 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087184    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb51b3d-3b09-4b5c-9a98-224123e80b50-config\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.087232 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txcr7\" (UniqueName: \"kubernetes.io/projected/fdb51b3d-3b09-4b5c-9a98-224123e80b50-kube-api-access-txcr7\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.087372 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087236    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087372 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.087372 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087372 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087347    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087373    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4705762e-ae4b-48d5-8fa1-a25b47b62d82-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.087546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.087671 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087570    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjt4n\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087671 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087615    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb51b3d-3b09-4b5c-9a98-224123e80b50-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.087671 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjtgp\" (UniqueName: \"kubernetes.io/projected/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-kube-api-access-vjtgp\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087679    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb51b3d-3b09-4b5c-9a98-224123e80b50-config\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbqhh\" (UniqueName: \"kubernetes.io/projected/7c943477-a00f-49fd-b404-20b64c51702b-kube-api-access-lbqhh\") pod \"volume-data-source-validator-5d456fdbcd-jjvr6\" (UID: \"7c943477-a00f-49fd-b404-20b64c51702b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087727    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.087753    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.087796 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.087756    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:25.088046 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.087813    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.587797597 +0000 UTC m=+63.849776439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:25.088046 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.088022    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.089610 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.089588    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb51b3d-3b09-4b5c-9a98-224123e80b50-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.098269 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.098246    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txcr7\" (UniqueName: \"kubernetes.io/projected/fdb51b3d-3b09-4b5c-9a98-224123e80b50-kube-api-access-txcr7\") pod \"service-ca-operator-5d46d9ffd4-sccbz\" (UID: \"fdb51b3d-3b09-4b5c-9a98-224123e80b50\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.099084 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.099063    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbqhh\" (UniqueName: \"kubernetes.io/projected/7c943477-a00f-49fd-b404-20b64c51702b-kube-api-access-lbqhh\") pod \"volume-data-source-validator-5d456fdbcd-jjvr6\" (UID: \"7c943477-a00f-49fd-b404-20b64c51702b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"
Jul 06 19:49:25.099344 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.099327    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjtgp\" (UniqueName: \"kubernetes.io/projected/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-kube-api-access-vjtgp\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.169266 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.169231    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"
Jul 06 19:49:25.188132 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188107    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188139    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188420 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188420 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.188269    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:25.188420 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.188284    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:25.188420 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.188350    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.688331298 +0000 UTC m=+63.950310161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:25.188609 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4705762e-ae4b-48d5-8fa1-a25b47b62d82-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.188609 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188609 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188542    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.188808 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjt4n\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.188808 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.188661    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.189066 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.188967    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:25.189066 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.189035    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.689017465 +0000 UTC m=+63.950996319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:25.189309 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.189289    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4705762e-ae4b-48d5-8fa1-a25b47b62d82-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.189368 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.189316    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.189368 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.189343    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.190852 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.190832    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.190942 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.190835    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.198692 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.198674    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.199216 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.199193    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjt4n\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.259338 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.259305    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"
Jul 06 19:49:25.280066 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.280040    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"
Jul 06 19:49:25.300060 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.299990    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86"]
Jul 06 19:49:25.303162 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:25.303135    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34011c5_eb88_4ca9_9b3f_26b0235ccc09.slice/crio-0340aedd0a7b1fe20214120541d87bba2151b55fc20d38cea79493ef36424714 WatchSource:0}: Error finding container 0340aedd0a7b1fe20214120541d87bba2151b55fc20d38cea79493ef36424714: Status 404 returned error can't find the container with id 0340aedd0a7b1fe20214120541d87bba2151b55fc20d38cea79493ef36424714
Jul 06 19:49:25.388973 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.388943    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz"]
Jul 06 19:49:25.391697 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:25.391671    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb51b3d_3b09_4b5c_9a98_224123e80b50.slice/crio-4dacdc539ba87050fa2529c6a45e5b3ab8d4d733ef37d1a655d9bbb654923746 WatchSource:0}: Error finding container 4dacdc539ba87050fa2529c6a45e5b3ab8d4d733ef37d1a655d9bbb654923746: Status 404 returned error can't find the container with id 4dacdc539ba87050fa2529c6a45e5b3ab8d4d733ef37d1a655d9bbb654923746
Jul 06 19:49:25.405220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.405198    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6"]
Jul 06 19:49:25.407967 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:25.407945    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c943477_a00f_49fd_b404_20b64c51702b.slice/crio-80f95d6d1883057c5dd4edfc8bc87bd20c3231e30ac53e3059ca70215deac890 WatchSource:0}: Error finding container 80f95d6d1883057c5dd4edfc8bc87bd20c3231e30ac53e3059ca70215deac890: Status 404 returned error can't find the container with id 80f95d6d1883057c5dd4edfc8bc87bd20c3231e30ac53e3059ca70215deac890
Jul 06 19:49:25.591324 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.591293    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:25.591491 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.591463    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:25.591561 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.591547    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:26.59152398 +0000 UTC m=+64.853502835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:25.692060 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.691980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:25.692208 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:25.692062    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:25.692208 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.692137    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:25.692208 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.692170    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:25.692208 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.692180    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:25.692404 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.692211    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:26.692191636 +0000 UTC m=+64.954170482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:25.692404 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:25.692230    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:26.692220049 +0000 UTC m=+64.954198898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:26.021236 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.021196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6" event={"ID":"7c943477-a00f-49fd-b404-20b64c51702b","Type":"ContainerStarted","Data":"80f95d6d1883057c5dd4edfc8bc87bd20c3231e30ac53e3059ca70215deac890"}
Jul 06 19:49:26.022394 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.022369    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz" event={"ID":"fdb51b3d-3b09-4b5c-9a98-224123e80b50","Type":"ContainerStarted","Data":"4dacdc539ba87050fa2529c6a45e5b3ab8d4d733ef37d1a655d9bbb654923746"}
Jul 06 19:49:26.023490 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.023456    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86" event={"ID":"a34011c5-eb88-4ca9-9b3f-26b0235ccc09","Type":"ContainerStarted","Data":"0340aedd0a7b1fe20214120541d87bba2151b55fc20d38cea79493ef36424714"}
Jul 06 19:49:26.600002 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.599960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:26.600002 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.599991    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:26.600234 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.600074    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.600051922 +0000 UTC m=+66.862030764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:26.700518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.700429    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:26.700666 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:26.700518    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:26.700666 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.700586    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:26.700666 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.700605    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:26.700666 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.700608    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:26.700666 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.700662    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.700643503 +0000 UTC m=+66.962622356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:26.700969 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:26.700688    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:28.700675758 +0000 UTC m=+66.962654608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:27.028557 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:27.028514    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6" event={"ID":"7c943477-a00f-49fd-b404-20b64c51702b","Type":"ContainerStarted","Data":"7f959cc75dff1038e3dac6e9527631e34fc3da36988a912a996da321617e7009"}
Jul 06 19:49:28.314370 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.314339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:49:28.314825 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.314452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:49:28.316887 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.316864    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 19:49:28.316887 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.316884    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 19:49:28.325280 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.325263    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 19:49:28.325350 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.325325    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs podName:dc51274b-629e-42ac-a025-e9ffafd363f5 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:32.325310929 +0000 UTC m=+130.587289770 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs") pod "network-metrics-daemon-pzghn" (UID: "dc51274b-629e-42ac-a025-e9ffafd363f5") : secret "metrics-daemon-secret" not found
Jul 06 19:49:28.327334 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.327314    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1605fa02-16bb-41b4-82fb-c934c9c21dc4-original-pull-secret\") pod \"global-pull-secret-syncer-sh2cx\" (UID: \"1605fa02-16bb-41b4-82fb-c934c9c21dc4\") " pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:49:28.415451 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.415382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:49:28.417905 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.417887    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 19:49:28.428589 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.428568    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 19:49:28.438839 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.438810    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc2dx\" (UniqueName: \"kubernetes.io/projected/a6290485-982b-4dd8-be92-f44b13ab3979-kube-api-access-mc2dx\") pod \"network-check-target-bth7t\" (UID: \"a6290485-982b-4dd8-be92-f44b13ab3979\") " pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:49:28.559164 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.559136    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh2cx"
Jul 06 19:49:28.567178 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.567152    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8fm7\""
Jul 06 19:49:28.575321 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.575302    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:49:28.618558 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.616555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:28.618558 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.616688    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:28.618558 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.616750    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.616732237 +0000 UTC m=+70.878711083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:28.708000 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.707944    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jjvr6" podStartSLOduration=3.728254203 podStartE2EDuration="4.707898287s" totalImagesPullingTime="979.644084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:27.044641091 +0000 UTC m=+65.306619956" watchObservedRunningTime="2026-07-06 19:49:28.707898287 +0000 UTC m=+66.969877143"
Jul 06 19:49:28.708303 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.708282    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sh2cx"]
Jul 06 19:49:28.711883 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:28.711845    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1605fa02_16bb_41b4_82fb_c934c9c21dc4.slice/crio-a15b03ba4bb8eff34211752a44c54a071040a8aa733b4a07427ea91b71821ece WatchSource:0}: Error finding container a15b03ba4bb8eff34211752a44c54a071040a8aa733b4a07427ea91b71821ece: Status 404 returned error can't find the container with id a15b03ba4bb8eff34211752a44c54a071040a8aa733b4a07427ea91b71821ece
Jul 06 19:49:28.717119 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.717098    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bth7t"]
Jul 06 19:49:28.717369 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.717351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:28.717454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:28.717440    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:28.717513 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.717487    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:28.717513 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.717505    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:28.717588 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.717532    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:28.717588 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.717559    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.717540683 +0000 UTC m=+70.979519540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:28.717588 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:28.717578    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.717566028 +0000 UTC m=+70.979544873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:28.720164 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:28.720142    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6290485_982b_4dd8_be92_f44b13ab3979.slice/crio-b40f947d61fcab4b1a1f41c6b776f49f569282e92524ce31753697baa40d0f0f WatchSource:0}: Error finding container b40f947d61fcab4b1a1f41c6b776f49f569282e92524ce31753697baa40d0f0f: Status 404 returned error can't find the container with id b40f947d61fcab4b1a1f41c6b776f49f569282e92524ce31753697baa40d0f0f
Jul 06 19:49:29.035076 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.035041    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz" event={"ID":"fdb51b3d-3b09-4b5c-9a98-224123e80b50","Type":"ContainerStarted","Data":"cd34477ffb74450bbee7afe976a8787a95c22e223491caa54cef4c23f86e4f26"}
Jul 06 19:49:29.036626 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.036595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86" event={"ID":"a34011c5-eb88-4ca9-9b3f-26b0235ccc09","Type":"ContainerStarted","Data":"bc090c41350830f3e295055dc02491da97548689474df93291c3a67e6d249263"}
Jul 06 19:49:29.038092 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.038071    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bth7t" event={"ID":"a6290485-982b-4dd8-be92-f44b13ab3979","Type":"ContainerStarted","Data":"b40f947d61fcab4b1a1f41c6b776f49f569282e92524ce31753697baa40d0f0f"}
Jul 06 19:49:29.039349 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.039327    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sh2cx" event={"ID":"1605fa02-16bb-41b4-82fb-c934c9c21dc4","Type":"ContainerStarted","Data":"a15b03ba4bb8eff34211752a44c54a071040a8aa733b4a07427ea91b71821ece"}
Jul 06 19:49:29.053208 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.053161    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz" podStartSLOduration=2.323884833 podStartE2EDuration="5.053144288s" totalImagesPullingTime="2.729259455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:29.052258899 +0000 UTC m=+67.314237764" watchObservedRunningTime="2026-07-06 19:49:29.053144288 +0000 UTC m=+67.315123152"
Jul 06 19:49:29.069886 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.069841    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86" podStartSLOduration=2.251017761 podStartE2EDuration="5.069823968s" totalImagesPullingTime="2.818806207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:29.069626242 +0000 UTC m=+67.331605107" watchObservedRunningTime="2026-07-06 19:49:29.069823968 +0000 UTC m=+67.331802812"
Jul 06 19:49:29.277829 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.277798    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5z8jt"]
Jul 06 19:49:29.280167 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.280143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.283550 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.283529    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 19:49:29.283664 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.283536    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 19:49:29.283884 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.283847    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 19:49:29.284049 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.284029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2mgfw\""
Jul 06 19:49:29.285685 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.285529    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 19:49:29.294720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.294701    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5z8jt"]
Jul 06 19:49:29.322505 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.322475    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-crio-socket\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.322907 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.322519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-data-volume\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.322907 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.322568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.322907 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.322602    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cbvp\" (UniqueName: \"kubernetes.io/projected/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-api-access-6cbvp\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.322907 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.322696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423229    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423256    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-crio-socket\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423280    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-data-volume\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423292    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423360    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423376    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:29.923352876 +0000 UTC m=+68.185331744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423303    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423454 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423422    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:29.92340668 +0000 UTC m=+68.185385523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : configmap "kube-rbac-proxy" not found
Jul 06 19:49:29.423845 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cbvp\" (UniqueName: \"kubernetes.io/projected/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-api-access-6cbvp\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423845 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.423667    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-crio-socket\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.423845 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423770    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:49:29.423845 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.423808    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls podName:52abd723-4417-452c-b72a-b3f5800f2762 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:01.423793909 +0000 UTC m=+99.685772764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls") pod "dns-default-24ntk" (UID: "52abd723-4417-452c-b72a-b3f5800f2762") : secret "dns-default-metrics-tls" not found
Jul 06 19:49:29.424143 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.424122    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-data-volume\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.442694 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.442664    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cbvp\" (UniqueName: \"kubernetes.io/projected/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-api-access-6cbvp\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.928957 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.928657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.928957 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:29.928757    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:29.928957 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.928871    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:49:29.928957 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.928886    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:29.928957 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.928956    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:30.928937263 +0000 UTC m=+69.190916107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:29.929353 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:29.928973    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:30.92896474 +0000 UTC m=+69.190943582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : configmap "kube-rbac-proxy" not found
Jul 06 19:49:30.936698 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:30.936262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:30.936698 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:30.936356    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:30.936698 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:30.936503    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:30.936698 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:30.936585    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.936565108 +0000 UTC m=+71.198543969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:30.936698 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:30.936702    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:49:30.937311 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:30.936751    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.936737345 +0000 UTC m=+71.198716186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : configmap "kube-rbac-proxy" not found
Jul 06 19:49:31.032381 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:31.032346    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42sf8_4a928635-994c-416e-aba8-35d28625546d/dns-node-resolver/0.log"
Jul 06 19:49:32.037148 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.037112    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-922hx_31a51b30-fae6-46a6-8202-a561d20de2e9/node-ca/0.log"
Jul 06 19:49:32.651258 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.651220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:32.651492 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.651472    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:32.651576 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.651544    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:40.651525319 +0000 UTC m=+78.913504180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:32.752198 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.752163    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:32.752375 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.752223    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:32.752375 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.752328    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:32.752375 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.752364    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:32.752525 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.752384    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:32.752525 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.752399    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:40.752379632 +0000 UTC m=+79.014358492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:32.752525 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.752434    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:40.752417415 +0000 UTC m=+79.014396278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:32.953937 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.953847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:32.954087 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:32.953954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:32.954087 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.953983    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:49:32.954087 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.954050    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:36.954031222 +0000 UTC m=+75.216010085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : configmap "kube-rbac-proxy" not found
Jul 06 19:49:32.954271 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.954122    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:32.954271 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:32.954197    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:36.954179906 +0000 UTC m=+75.216158760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:33.232700 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:33.232624    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tsgnw_9731f41f-d036-4b90-a4ec-b6963f26c4cb/serve-healthcheck-canary/0.log"
Jul 06 19:49:34.055294 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:34.055258    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bth7t" event={"ID":"a6290485-982b-4dd8-be92-f44b13ab3979","Type":"ContainerStarted","Data":"9a587a8399179270d1e7aecd8e1116997848960a2a0fbf98304d9fde1f6ce8f2"}
Jul 06 19:49:34.055487 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:34.055398    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:49:34.056808 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:34.056783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sh2cx" event={"ID":"1605fa02-16bb-41b4-82fb-c934c9c21dc4","Type":"ContainerStarted","Data":"fba4a030f9884426f94677813e32d8529e97e8437c4d3af50da34f6d7bd5b0c2"}
Jul 06 19:49:34.071683 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:34.071639    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bth7t" podStartSLOduration=66.037274979 podStartE2EDuration="1m11.071623614s" totalImagesPullingTime="5.034348635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:34.071539349 +0000 UTC m=+72.333518214" watchObservedRunningTime="2026-07-06 19:49:34.071623614 +0000 UTC m=+72.333602483"
Jul 06 19:49:36.988101 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:36.988061    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:36.988568 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:36.988126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:36.988568 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:36.988210    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:49:36.988568 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:36.988238    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:36.988568 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:36.988278    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:44.988260763 +0000 UTC m=+83.250239618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : configmap "kube-rbac-proxy" not found
Jul 06 19:49:36.988568 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:36.988294    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:44.988287994 +0000 UTC m=+83.250266836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:40.717078 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:40.717035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:40.717477 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.717188    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:40.717477 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.717265    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls podName:d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:56.7172452 +0000 UTC m=+94.979224043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m7hc7" (UID: "d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0") : secret "cluster-monitoring-operator-tls" not found
Jul 06 19:49:40.818098 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:40.818057    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:40.818287 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:40.818111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:40.818287 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.818235    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 19:49:40.818388 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.818328    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert podName:4705762e-ae4b-48d5-8fa1-a25b47b62d82 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:56.818305262 +0000 UTC m=+95.080284113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-sg4dn" (UID: "4705762e-ae4b-48d5-8fa1-a25b47b62d82") : secret "networking-console-plugin-cert" not found
Jul 06 19:49:40.818431 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.818400    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 19:49:40.818431 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.818417    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-945cb6bd5-l9kkn: secret "image-registry-tls" not found
Jul 06 19:49:40.818495 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:40.818468    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls podName:72ef9420-f133-4d3a-a4b2-7575b495af0e nodeName:}" failed. No retries permitted until 2026-07-06 19:49:56.81845534 +0000 UTC m=+95.080434182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls") pod "image-registry-945cb6bd5-l9kkn" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e") : secret "image-registry-tls" not found
Jul 06 19:49:45.050162 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:45.050022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:45.050162 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:45.050097    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:45.050593 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:45.050189    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:45.050593 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:49:45.050280    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls podName:1a6bfb02-7938-4c5f-a249-cfcd5b428ad4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:01.050256925 +0000 UTC m=+99.312235984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z8jt" (UID: "1a6bfb02-7938-4c5f-a249-cfcd5b428ad4") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:49:45.050672 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:45.050622    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:49:56.739096 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.739063    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:56.741386 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.741367    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m7hc7\" (UID: \"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:56.786385 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.786365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"
Jul 06 19:49:56.840491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.840440    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:56.840624 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.840517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:56.843155 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.843121    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"image-registry-945cb6bd5-l9kkn\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") " pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:56.843582 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.843559    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4705762e-ae4b-48d5-8fa1-a25b47b62d82-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-sg4dn\" (UID: \"4705762e-ae4b-48d5-8fa1-a25b47b62d82\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:56.860003 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.859982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:56.908700 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.908640    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sh2cx" podStartSLOduration=88.849511301 podStartE2EDuration="1m33.908591968s" totalImagesPullingTime="5.059080667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:34.086637038 +0000 UTC m=+72.348615905" watchObservedRunningTime="2026-07-06 19:49:56.908591968 +0000 UTC m=+95.170570833"
Jul 06 19:49:56.910420 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.910393    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7"]
Jul 06 19:49:56.985988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:56.985967    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:49:56.987082 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:56.987057    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ef9420_f133_4d3a_a4b2_7575b495af0e.slice/crio-07f80e56d567328ee6413e0808c229757b3e999eb067abf2f44b1d7c66111a51 WatchSource:0}: Error finding container 07f80e56d567328ee6413e0808c229757b3e999eb067abf2f44b1d7c66111a51: Status 404 returned error can't find the container with id 07f80e56d567328ee6413e0808c229757b3e999eb067abf2f44b1d7c66111a51
Jul 06 19:49:57.106238 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.106182    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"
Jul 06 19:49:57.113489 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.112499    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"]
Jul 06 19:49:57.115353 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.115334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.119144 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.119116    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 19:49:57.119247 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.119155    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 19:49:57.119809 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.119780    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 19:49:57.119809 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.119800    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 19:49:57.119981 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.119948    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 19:49:57.120052 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.120036    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 19:49:57.128061 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.127930    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 19:49:57.132092 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.131588    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" event={"ID":"72ef9420-f133-4d3a-a4b2-7575b495af0e","Type":"ContainerStarted","Data":"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"}
Jul 06 19:49:57.132092 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.131673    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" event={"ID":"72ef9420-f133-4d3a-a4b2-7575b495af0e","Type":"ContainerStarted","Data":"07f80e56d567328ee6413e0808c229757b3e999eb067abf2f44b1d7c66111a51"}
Jul 06 19:49:57.132092 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.131718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:49:57.133169 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.133126    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"]
Jul 06 19:49:57.133961 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.133937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7" event={"ID":"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0","Type":"ContainerStarted","Data":"d526268d5889f7b65fbfdc4fc338ae59499332639364684095845de4efda574b"}
Jul 06 19:49:57.181605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.181561    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" podStartSLOduration=32.181545028 podStartE2EDuration="32.181545028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:57.181064313 +0000 UTC m=+95.443043188" watchObservedRunningTime="2026-07-06 19:49:57.181545028 +0000 UTC m=+95.443523891"
Jul 06 19:49:57.195668 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.195639    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:49:57.243400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.243542 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243405    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf95d751-b134-4294-8268-a63beea2e9f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.243661 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243638    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.243725 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243711    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.243849 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.243945 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.243927    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhk7m\" (UniqueName: \"kubernetes.io/projected/cf95d751-b134-4294-8268-a63beea2e9f9-kube-api-access-hhk7m\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.265157 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.265130    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn"]
Jul 06 19:49:57.267777 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:57.267749    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4705762e_ae4b_48d5_8fa1_a25b47b62d82.slice/crio-c98b8a156d94fa735ee594d0167f0b4a27ff146dd7367e6e46580526822f45c5 WatchSource:0}: Error finding container c98b8a156d94fa735ee594d0167f0b4a27ff146dd7367e6e46580526822f45c5: Status 404 returned error can't find the container with id c98b8a156d94fa735ee594d0167f0b4a27ff146dd7367e6e46580526822f45c5
Jul 06 19:49:57.344985 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.344950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.345142 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.344998    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.345142 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.345033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.345142 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.345053    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhk7m\" (UniqueName: \"kubernetes.io/projected/cf95d751-b134-4294-8268-a63beea2e9f9-kube-api-access-hhk7m\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.345142 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.345080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.345332 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.345266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf95d751-b134-4294-8268-a63beea2e9f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.346060 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.346030    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf95d751-b134-4294-8268-a63beea2e9f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.347647 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.347623    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-ca\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.347794 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.347773    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.347938 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.347901    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.347981 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.347950    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cf95d751-b134-4294-8268-a63beea2e9f9-hub\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.353002 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.352979    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhk7m\" (UniqueName: \"kubernetes.io/projected/cf95d751-b134-4294-8268-a63beea2e9f9-kube-api-access-hhk7m\") pod \"cluster-proxy-proxy-agent-77797b89c6-z6q6g\" (UID: \"cf95d751-b134-4294-8268-a63beea2e9f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.437455 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.437430    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"
Jul 06 19:49:57.572542 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:57.572513    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g"]
Jul 06 19:49:57.575671 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:49:57.575644    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf95d751_b134_4294_8268_a63beea2e9f9.slice/crio-fa380fb25ad40b9312326053bf859dafc335461a79875fa7f06e35a6db74be89 WatchSource:0}: Error finding container fa380fb25ad40b9312326053bf859dafc335461a79875fa7f06e35a6db74be89: Status 404 returned error can't find the container with id fa380fb25ad40b9312326053bf859dafc335461a79875fa7f06e35a6db74be89
Jul 06 19:49:58.139496 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:58.139455    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g" event={"ID":"cf95d751-b134-4294-8268-a63beea2e9f9","Type":"ContainerStarted","Data":"fa380fb25ad40b9312326053bf859dafc335461a79875fa7f06e35a6db74be89"}
Jul 06 19:49:58.141197 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:58.141139    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn" event={"ID":"4705762e-ae4b-48d5-8fa1-a25b47b62d82","Type":"ContainerStarted","Data":"c98b8a156d94fa735ee594d0167f0b4a27ff146dd7367e6e46580526822f45c5"}
Jul 06 19:49:59.147946 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:59.147882    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn" event={"ID":"4705762e-ae4b-48d5-8fa1-a25b47b62d82","Type":"ContainerStarted","Data":"7ddba9e0dbfcccad408df087e78214fd16a4f9748eaae6be5795cedaa63545af"}
Jul 06 19:49:59.153173 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:59.152941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7" event={"ID":"d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0","Type":"ContainerStarted","Data":"cb5e5cdc04fd15ba7e2ea441e9c60b99894e337c12992ff091185795cfb34aa0"}
Jul 06 19:49:59.164370 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:59.164296    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-sg4dn" podStartSLOduration=33.425310813 podStartE2EDuration="35.164278932s" totalImagesPullingTime="1.738968119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:59.163513073 +0000 UTC m=+97.425491939" watchObservedRunningTime="2026-07-06 19:49:59.164278932 +0000 UTC m=+97.426257806"
Jul 06 19:49:59.200924 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:49:59.200100    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m7hc7" podStartSLOduration=33.10754538 podStartE2EDuration="35.200079487s" totalImagesPullingTime="2.092534107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:59.199817446 +0000 UTC m=+97.461796310" watchObservedRunningTime="2026-07-06 19:49:59.200079487 +0000 UTC m=+97.462058356"
Jul 06 19:50:01.080550 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.080509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:50:01.082958 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.082928    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6bfb02-7938-4c5f-a249-cfcd5b428ad4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z8jt\" (UID: \"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4\") " pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:50:01.093811 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.093787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5z8jt"
Jul 06 19:50:01.484464 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.484379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:50:01.486884 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.486861    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52abd723-4417-452c-b72a-b3f5800f2762-metrics-tls\") pod \"dns-default-24ntk\" (UID: \"52abd723-4417-452c-b72a-b3f5800f2762\") " pod="openshift-dns/dns-default-24ntk"
Jul 06 19:50:01.502576 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.502555    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qb4h6\""
Jul 06 19:50:01.510832 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:01.510808    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-24ntk"
Jul 06 19:50:02.169957 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:02.169926    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5z8jt"]
Jul 06 19:50:02.172833 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:50:02.172807    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6bfb02_7938_4c5f_a249_cfcd5b428ad4.slice/crio-1d005777a82401b9fcfebcfd443bb43ba298216598c7cd568eab68daf3fd96b3 WatchSource:0}: Error finding container 1d005777a82401b9fcfebcfd443bb43ba298216598c7cd568eab68daf3fd96b3: Status 404 returned error can't find the container with id 1d005777a82401b9fcfebcfd443bb43ba298216598c7cd568eab68daf3fd96b3
Jul 06 19:50:02.183360 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:02.183336    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-24ntk"]
Jul 06 19:50:02.185719 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:50:02.185693    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52abd723_4417_452c_b72a_b3f5800f2762.slice/crio-ba887ee345d8fc05357ea05eca2e653faf722c5933665a89e62f8156007750cc WatchSource:0}: Error finding container ba887ee345d8fc05357ea05eca2e653faf722c5933665a89e62f8156007750cc: Status 404 returned error can't find the container with id ba887ee345d8fc05357ea05eca2e653faf722c5933665a89e62f8156007750cc
Jul 06 19:50:03.178263 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.178214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g" event={"ID":"cf95d751-b134-4294-8268-a63beea2e9f9","Type":"ContainerStarted","Data":"eeaa3aa9b7b55223d0b31713fcf96f3982b2b0cc0d0cd4cb52e6c26bb36fa1db"}
Jul 06 19:50:03.178861 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.178840    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g" event={"ID":"cf95d751-b134-4294-8268-a63beea2e9f9","Type":"ContainerStarted","Data":"391b201ec38d55a71b5dc17968687cbebc70cb0edb8b74c4181ba4a05dd0ba68"}
Jul 06 19:50:03.179016 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.179000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g" event={"ID":"cf95d751-b134-4294-8268-a63beea2e9f9","Type":"ContainerStarted","Data":"2512461cc9293485f576ddcbc1727066ac66056a37fe77bf74aca7286f782ef5"}
Jul 06 19:50:03.180724 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.180655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ntk" event={"ID":"52abd723-4417-452c-b72a-b3f5800f2762","Type":"ContainerStarted","Data":"ba887ee345d8fc05357ea05eca2e653faf722c5933665a89e62f8156007750cc"}
Jul 06 19:50:03.182192 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.182170    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z8jt" event={"ID":"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4","Type":"ContainerStarted","Data":"1d005777a82401b9fcfebcfd443bb43ba298216598c7cd568eab68daf3fd96b3"}
Jul 06 19:50:03.200439 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:03.200072    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77797b89c6-z6q6g" podStartSLOduration=1.3344820450000001 podStartE2EDuration="6.200053705s" totalImagesPullingTime="4.86557166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:03.198289056 +0000 UTC m=+101.460267924" watchObservedRunningTime="2026-07-06 19:50:03.200053705 +0000 UTC m=+101.462032570"
Jul 06 19:50:05.063387 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:05.063349    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bth7t"
Jul 06 19:50:07.198002 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:07.197966    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ntk" event={"ID":"52abd723-4417-452c-b72a-b3f5800f2762","Type":"ContainerStarted","Data":"7dd659d443240f8f84342e9c996d5e8161139e6daf62937fb45e6cc5cee98ba7"}
Jul 06 19:50:07.198476 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:07.198029    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ntk" event={"ID":"52abd723-4417-452c-b72a-b3f5800f2762","Type":"ContainerStarted","Data":"3226bb8474ca4761dfede1636a4e22de27d705965bde049ea715c3f2aac306f9"}
Jul 06 19:50:07.198476 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:07.198061    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-24ntk"
Jul 06 19:50:07.199624 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:07.199593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z8jt" event={"ID":"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4","Type":"ContainerStarted","Data":"343c3503438067d79a76cde3f5d8eec450a3234fe5457483b87d6afd60ddacd8"}
Jul 06 19:50:07.218245 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:07.218193    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-24ntk" podStartSLOduration=65.725793615 podStartE2EDuration="1m10.218177056s" totalImagesPullingTime="4.492383441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:07.216938201 +0000 UTC m=+105.478917071" watchObservedRunningTime="2026-07-06 19:50:07.218177056 +0000 UTC m=+105.480155919"
Jul 06 19:50:08.205076 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:08.205036    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z8jt" event={"ID":"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4","Type":"ContainerStarted","Data":"04de9e526693e78446b7831b073e3dcd25763f2e04a63e360dfdaab5bd64143f"}
Jul 06 19:50:09.978153 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.978121    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-97p8z"]
Jul 06 19:50:09.987705 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.987679    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:09.990023 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.989999    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 19:50:09.990289 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.990276    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 19:50:09.990391 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.990301    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 19:50:09.990391 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.990346    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 19:50:09.991351 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:09.991336    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s657x\""
Jul 06 19:50:10.154437 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154411    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rnjj\" (UniqueName: \"kubernetes.io/projected/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-kube-api-access-8rnjj\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-accelerators-collector-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-root\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154558    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-textfile\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-metrics-client-ca\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-sys\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.154763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.154713    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-wtmp\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.215283 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.215247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z8jt" event={"ID":"1a6bfb02-7938-4c5f-a249-cfcd5b428ad4","Type":"ContainerStarted","Data":"7446b0386042a30988bdb821c09fe70f9d9618769308ef91ddad4b8992bb2085"}
Jul 06 19:50:10.235012 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.234939    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5z8jt" podStartSLOduration=33.497776711 podStartE2EDuration="41.234925129s" totalImagesPullingTime="7.737148418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:49:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:10.233895821 +0000 UTC m=+108.495874687" watchObservedRunningTime="2026-07-06 19:50:10.234925129 +0000 UTC m=+108.496904024"
Jul 06 19:50:10.255479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-metrics-client-ca\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-sys\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-wtmp\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rnjj\" (UniqueName: \"kubernetes.io/projected/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-kube-api-access-8rnjj\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255573    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255584    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-sys\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255610    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-accelerators-collector-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-root\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:50:10.255644    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255689    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-wtmp\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:50:10.255702    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls podName:c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:10.755686207 +0000 UTC m=+109.017665063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls") pod "node-exporter-97p8z" (UID: "c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4") : secret "node-exporter-tls" not found
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255701    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-root\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.255758 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255647    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-textfile\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.256168 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.255974    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-textfile\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.256168 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.256146    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-metrics-client-ca\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.256232 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.256181    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-accelerators-collector-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.257741 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.257724    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.263859 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.263837    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rnjj\" (UniqueName: \"kubernetes.io/projected/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-kube-api-access-8rnjj\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.759382 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.759343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.761635 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.761603    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4-node-exporter-tls\") pod \"node-exporter-97p8z\" (UID: \"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4\") " pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.897643 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:10.897613    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-97p8z"
Jul 06 19:50:10.911605 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:50:10.911579    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc939f3d9_44f1_432c_bc4e_f4e9bd51ddb4.slice/crio-b6f41c64bf93c730f765000aa3a91123be354c598dce7e3ae211ac1cae608e69 WatchSource:0}: Error finding container b6f41c64bf93c730f765000aa3a91123be354c598dce7e3ae211ac1cae608e69: Status 404 returned error can't find the container with id b6f41c64bf93c730f765000aa3a91123be354c598dce7e3ae211ac1cae608e69
Jul 06 19:50:11.219468 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:11.219377    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97p8z" event={"ID":"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4","Type":"ContainerStarted","Data":"b6f41c64bf93c730f765000aa3a91123be354c598dce7e3ae211ac1cae608e69"}
Jul 06 19:50:12.223739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:12.223709    2565 generic.go:356] "Generic (PLEG): container finished" podID="c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4" containerID="ef0eb6e28094e8ced66382a3c8f38748626a605a2b400129300afeee6098bd94" exitCode=0
Jul 06 19:50:12.224049 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:12.223745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97p8z" event={"ID":"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4","Type":"ContainerDied","Data":"ef0eb6e28094e8ced66382a3c8f38748626a605a2b400129300afeee6098bd94"}
Jul 06 19:50:13.229853 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:13.229816    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97p8z" event={"ID":"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4","Type":"ContainerStarted","Data":"2331ce99d2c1a8bd8e4f2be58f491e20df28f1b9782736b9157f74932cae8343"}
Jul 06 19:50:13.230354 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:13.229868    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97p8z" event={"ID":"c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4","Type":"ContainerStarted","Data":"ff7323fb368fdfc2d388a29c9fdd2fc9975ce82f32577145d484175c5b05d662"}
Jul 06 19:50:13.250444 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:13.250399    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-97p8z" podStartSLOduration=3.038518081 podStartE2EDuration="4.250385036s" totalImagesPullingTime="1.146847954s" totalInitContainerRuntime="65.019001ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:13.24879031 +0000 UTC m=+111.510769174" watchObservedRunningTime="2026-07-06 19:50:13.250385036 +0000 UTC m=+111.512363900"
Jul 06 19:50:17.207492 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:17.207456    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-24ntk"
Jul 06 19:50:19.158630 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:19.158604    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:50:24.178228 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.178162    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" podUID="72ef9420-f133-4d3a-a4b2-7575b495af0e" containerName="registry" containerID="cri-o://b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693" gracePeriod=30
Jul 06 19:50:24.416211 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.416185    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:50:24.467018 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.466940    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467018 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.466988    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467018 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467016    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467056    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467091    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjt4n\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467240    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467272    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467306    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") pod \"72ef9420-f133-4d3a-a4b2-7575b495af0e\" (UID: \"72ef9420-f133-4d3a-a4b2-7575b495af0e\") "
Jul 06 19:50:24.467415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467385    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:50:24.467757 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.467707    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:50:24.468301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.468264    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-certificates\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.468301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.468297    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72ef9420-f133-4d3a-a4b2-7575b495af0e-trusted-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.469739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.469691    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:24.469739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.469714    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n" (OuterVolumeSpecName: "kube-api-access-xjt4n") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "kube-api-access-xjt4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:24.469739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.469718    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:50:24.469930 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.469770    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:50:24.469930 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.469866    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:24.475720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.475698    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "72ef9420-f133-4d3a-a4b2-7575b495af0e" (UID: "72ef9420-f133-4d3a-a4b2-7575b495af0e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:24.569396 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569373    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-installation-pull-secrets\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.569396 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569395    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjt4n\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-kube-api-access-xjt4n\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.569518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569405    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-bound-sa-token\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.569518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569414    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72ef9420-f133-4d3a-a4b2-7575b495af0e-registry-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.569518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569422    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72ef9420-f133-4d3a-a4b2-7575b495af0e-ca-trust-extracted\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:24.569518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:24.569431    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72ef9420-f133-4d3a-a4b2-7575b495af0e-image-registry-private-configuration\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:50:25.270893 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.270863    2565 generic.go:356] "Generic (PLEG): container finished" podID="72ef9420-f133-4d3a-a4b2-7575b495af0e" containerID="b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693" exitCode=0
Jul 06 19:50:25.271336 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.270935    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn"
Jul 06 19:50:25.271336 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.270956    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" event={"ID":"72ef9420-f133-4d3a-a4b2-7575b495af0e","Type":"ContainerDied","Data":"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"}
Jul 06 19:50:25.271336 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.271013    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-945cb6bd5-l9kkn" event={"ID":"72ef9420-f133-4d3a-a4b2-7575b495af0e","Type":"ContainerDied","Data":"07f80e56d567328ee6413e0808c229757b3e999eb067abf2f44b1d7c66111a51"}
Jul 06 19:50:25.271336 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.271039    2565 scope.go:117] "RemoveContainer" containerID="b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"
Jul 06 19:50:25.279052 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.279037    2565 scope.go:117] "RemoveContainer" containerID="b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"
Jul 06 19:50:25.279294 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:50:25.279274    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693\": container with ID starting with b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693 not found: ID does not exist" containerID="b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"
Jul 06 19:50:25.279334 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.279307    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693"} err="failed to get container status \"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693\": rpc error: code = NotFound desc = could not find container \"b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693\": container with ID starting with b53996862d561d8ddf571e3e3958a446a1b5a9536d06947f540515fc0cad7693 not found: ID does not exist"
Jul 06 19:50:25.290825 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.290803    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:50:25.301301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:25.301214    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-945cb6bd5-l9kkn"]
Jul 06 19:50:26.653654 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:26.653623    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72ef9420-f133-4d3a-a4b2-7575b495af0e" path="/var/lib/kubelet/pods/72ef9420-f133-4d3a-a4b2-7575b495af0e/volumes"
Jul 06 19:50:32.423823 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:32.423786    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:50:32.426007 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:32.425991    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc51274b-629e-42ac-a025-e9ffafd363f5-metrics-certs\") pod \"network-metrics-daemon-pzghn\" (UID: \"dc51274b-629e-42ac-a025-e9ffafd363f5\") " pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:50:32.472268 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:32.472247    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xsp7\""
Jul 06 19:50:32.480523 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:32.480505    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzghn"
Jul 06 19:50:32.699045 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:32.698970    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pzghn"]
Jul 06 19:50:32.701491 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:50:32.701464    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc51274b_629e_42ac_a025_e9ffafd363f5.slice/crio-82b0de736fee26efb4d8ff1da63982777b571a9b66077457f175d5d16b506784 WatchSource:0}: Error finding container 82b0de736fee26efb4d8ff1da63982777b571a9b66077457f175d5d16b506784: Status 404 returned error can't find the container with id 82b0de736fee26efb4d8ff1da63982777b571a9b66077457f175d5d16b506784
Jul 06 19:50:33.299339 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:33.299298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzghn" event={"ID":"dc51274b-629e-42ac-a025-e9ffafd363f5","Type":"ContainerStarted","Data":"82b0de736fee26efb4d8ff1da63982777b571a9b66077457f175d5d16b506784"}
Jul 06 19:50:34.304352 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:34.304321    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzghn" event={"ID":"dc51274b-629e-42ac-a025-e9ffafd363f5","Type":"ContainerStarted","Data":"a4f0e4b4183fb5590d99df529e8137d5b4f64439161ab9889c32d17bff0a42bf"}
Jul 06 19:50:34.304688 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:34.304373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzghn" event={"ID":"dc51274b-629e-42ac-a025-e9ffafd363f5","Type":"ContainerStarted","Data":"c2486761d8e318385d6528b224e4c3353b4d037a194cc7e273b41bdbb05b13af"}
Jul 06 19:50:34.322719 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:34.322675    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pzghn" podStartSLOduration=130.037909459 podStartE2EDuration="2m11.322659689s" totalImagesPullingTime="1.28475023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:34.321736516 +0000 UTC m=+132.583715379" watchObservedRunningTime="2026-07-06 19:50:34.322659689 +0000 UTC m=+132.584638552"
Jul 06 19:50:40.728984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.728950    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:50:40.729455 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.729182    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72ef9420-f133-4d3a-a4b2-7575b495af0e" containerName="registry"
Jul 06 19:50:40.729455 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.729193    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="72ef9420-f133-4d3a-a4b2-7575b495af0e" containerName="registry"
Jul 06 19:50:40.729455 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.729241    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="72ef9420-f133-4d3a-a4b2-7575b495af0e" containerName="registry"
Jul 06 19:50:40.742731 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.742705    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:50:40.742860 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.742802    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.745302 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.745275    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 19:50:40.745436 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.745409    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 19:50:40.745508 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.745484    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9f9bq\""
Jul 06 19:50:40.745599 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.745575    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 19:50:40.746649 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.746630    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 19:50:40.746649 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.746636    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 19:50:40.746802 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.746695    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 19:50:40.746994 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.746979    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 19:50:40.750424 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.750402    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 19:50:40.781775 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.781934 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781788    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.781934 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.781934 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.782062 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781962    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.782062 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.781989    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6sv5\" (UniqueName: \"kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.782062 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.782013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883128 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883128 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6sv5\" (UniqueName: \"kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883184    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883356 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883260    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883581 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883481    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.883986 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.883959    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.884123 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.884006    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.884192 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.884128    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.884192 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.884151    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.885774 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.885746    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.885867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.885852    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:40.892741 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:40.892716    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6sv5\" (UniqueName: \"kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5\") pod \"console-66b6498464-55qls\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") " pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:41.054322 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:41.054296    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:41.179705 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:41.179677    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:50:41.328248 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:41.328168    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6498464-55qls" event={"ID":"25b992d4-b989-49f6-8209-ce868734fa8b","Type":"ContainerStarted","Data":"9f354b7ac83e3e1f8df33f19f30c56b7a344f2844f202b6e8f4b375d9f2bb007"}
Jul 06 19:50:44.341882 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:44.341847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6498464-55qls" event={"ID":"25b992d4-b989-49f6-8209-ce868734fa8b","Type":"ContainerStarted","Data":"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"}
Jul 06 19:50:44.366477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:44.366433    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b6498464-55qls" podStartSLOduration=1.577023984 podStartE2EDuration="4.366411129s" totalImagesPullingTime="2.789387145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:44.366056895 +0000 UTC m=+142.628035758" watchObservedRunningTime="2026-07-06 19:50:44.366411129 +0000 UTC m=+142.628389992"
Jul 06 19:50:51.055292 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:51.055250    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:51.055710 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:51.055305    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:51.060172 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:51.060149    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:51.368967 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:51.368877    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:50:59.393312 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:59.393278    2565 generic.go:356] "Generic (PLEG): container finished" podID="a34011c5-eb88-4ca9-9b3f-26b0235ccc09" containerID="bc090c41350830f3e295055dc02491da97548689474df93291c3a67e6d249263" exitCode=0
Jul 06 19:50:59.393697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:59.393330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86" event={"ID":"a34011c5-eb88-4ca9-9b3f-26b0235ccc09","Type":"ContainerDied","Data":"bc090c41350830f3e295055dc02491da97548689474df93291c3a67e6d249263"}
Jul 06 19:50:59.393697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:50:59.393635    2565 scope.go:117] "RemoveContainer" containerID="bc090c41350830f3e295055dc02491da97548689474df93291c3a67e6d249263"
Jul 06 19:51:00.399109 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:00.399075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hjc86" event={"ID":"a34011c5-eb88-4ca9-9b3f-26b0235ccc09","Type":"ContainerStarted","Data":"598b4bdaff694e233069869652bfd93bb8a4aef59d8136b50302807378f5b08f"}
Jul 06 19:51:05.416711 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:05.416679    2565 generic.go:356] "Generic (PLEG): container finished" podID="fdb51b3d-3b09-4b5c-9a98-224123e80b50" containerID="cd34477ffb74450bbee7afe976a8787a95c22e223491caa54cef4c23f86e4f26" exitCode=0
Jul 06 19:51:05.417125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:05.416730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz" event={"ID":"fdb51b3d-3b09-4b5c-9a98-224123e80b50","Type":"ContainerDied","Data":"cd34477ffb74450bbee7afe976a8787a95c22e223491caa54cef4c23f86e4f26"}
Jul 06 19:51:05.417125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:05.417040    2565 scope.go:117] "RemoveContainer" containerID="cd34477ffb74450bbee7afe976a8787a95c22e223491caa54cef4c23f86e4f26"
Jul 06 19:51:06.422195 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:06.422157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-sccbz" event={"ID":"fdb51b3d-3b09-4b5c-9a98-224123e80b50","Type":"ContainerStarted","Data":"2ac733004dd13fdbc2b29738d208bec584a6fe2439fb4d4a9a29df5e003f26b0"}
Jul 06 19:51:37.290763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:51:37.290729    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:52:02.310820 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.310775    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66b6498464-55qls" podUID="25b992d4-b989-49f6-8209-ce868734fa8b" containerName="console" containerID="cri-o://e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc" gracePeriod=15
Jul 06 19:52:02.543552 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.543521    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:52:02.608115 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.608030    2565 generic.go:356] "Generic (PLEG): container finished" podID="25b992d4-b989-49f6-8209-ce868734fa8b" containerID="e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc" exitCode=0
Jul 06 19:52:02.608115 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.608074    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6498464-55qls"
Jul 06 19:52:02.608115 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.608089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6498464-55qls" event={"ID":"25b992d4-b989-49f6-8209-ce868734fa8b","Type":"ContainerDied","Data":"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"}
Jul 06 19:52:02.608337 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.608166    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6498464-55qls" event={"ID":"25b992d4-b989-49f6-8209-ce868734fa8b","Type":"ContainerDied","Data":"9f354b7ac83e3e1f8df33f19f30c56b7a344f2844f202b6e8f4b375d9f2bb007"}
Jul 06 19:52:02.608337 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.608192    2565 scope.go:117] "RemoveContainer" containerID="e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"
Jul 06 19:52:02.615676 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.615656    2565 scope.go:117] "RemoveContainer" containerID="e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"
Jul 06 19:52:02.616015 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:52:02.615906    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc\": container with ID starting with e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc not found: ID does not exist" containerID="e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"
Jul 06 19:52:02.616068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.616026    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc"} err="failed to get container status \"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc\": rpc error: code = NotFound desc = could not find container \"e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc\": container with ID starting with e9f1b666e0c204887294d5bb6be705e6ef56d2671ea751a49e401a7524729acc not found: ID does not exist"
Jul 06 19:52:02.734493 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734464    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734501    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734520    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734547    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734580    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734624    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.734648 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734643    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6sv5\" (UniqueName: \"kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5\") pod \"25b992d4-b989-49f6-8209-ce868734fa8b\" (UID: \"25b992d4-b989-49f6-8209-ce868734fa8b\") "
Jul 06 19:52:02.735017 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.734857    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca" (OuterVolumeSpecName: "service-ca") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:52:02.735153 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.735129    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:52:02.735215 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.735175    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:52:02.735252 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.735178    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config" (OuterVolumeSpecName: "console-config") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:52:02.736875 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.736843    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:52:02.736875 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.736862    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:52:02.737082 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.736904    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5" (OuterVolumeSpecName: "kube-api-access-g6sv5") pod "25b992d4-b989-49f6-8209-ce868734fa8b" (UID: "25b992d4-b989-49f6-8209-ce868734fa8b"). InnerVolumeSpecName "kube-api-access-g6sv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:52:02.835992 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.835954    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-serving-cert\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.835992 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.835986    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.836177 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.836001    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-oauth-serving-cert\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.836177 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.836014    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b992d4-b989-49f6-8209-ce868734fa8b-console-oauth-config\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.836177 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.836025    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-trusted-ca-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.836177 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.836036    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b992d4-b989-49f6-8209-ce868734fa8b-console-config\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.836177 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.836048    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6sv5\" (UniqueName: \"kubernetes.io/projected/25b992d4-b989-49f6-8209-ce868734fa8b-kube-api-access-g6sv5\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:02.928716 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.928690    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:52:02.932300 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:02.932271    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b6498464-55qls"]
Jul 06 19:52:04.653657 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:04.653627    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25b992d4-b989-49f6-8209-ce868734fa8b" path="/var/lib/kubelet/pods/25b992d4-b989-49f6-8209-ce868734fa8b/volumes"
Jul 06 19:52:12.181000 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.180967    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"]
Jul 06 19:52:12.181461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.181201    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25b992d4-b989-49f6-8209-ce868734fa8b" containerName="console"
Jul 06 19:52:12.181461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.181212    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="25b992d4-b989-49f6-8209-ce868734fa8b" containerName="console"
Jul 06 19:52:12.181461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.181254    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="25b992d4-b989-49f6-8209-ce868734fa8b" containerName="console"
Jul 06 19:52:12.185186 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.185168    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.187725 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.187703    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:52:12.188925 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.188896    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:52:12.189016 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.188948    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8tsp2\""
Jul 06 19:52:12.193115 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.193096    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"]
Jul 06 19:52:12.202665 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.202646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.202770 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.202674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpxhn\" (UniqueName: \"kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.202770 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.202696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.303599 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.303572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.303722 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.303602    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpxhn\" (UniqueName: \"kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.303722 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.303623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.303968 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.303948    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.304030 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.303969    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.311393 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.311371    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpxhn\" (UniqueName: \"kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.494602 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.494545    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:12.617098 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.617069    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"]
Jul 06 19:52:12.620223 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:52:12.620197    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b7329c_2031_4869_96fd_de225ca919a1.slice/crio-b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8 WatchSource:0}: Error finding container b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8: Status 404 returned error can't find the container with id b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8
Jul 06 19:52:12.641385 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:12.641348    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerStarted","Data":"b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8"}
Jul 06 19:52:20.672802 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:20.672767    2565 generic.go:356] "Generic (PLEG): container finished" podID="16b7329c-2031-4869-96fd-de225ca919a1" containerID="85a38900ec165a60339913c770027c20264d4cb6eee4f1745ef1a1827bcd508b" exitCode=0
Jul 06 19:52:20.673166 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:20.672833    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerDied","Data":"85a38900ec165a60339913c770027c20264d4cb6eee4f1745ef1a1827bcd508b"}
Jul 06 19:52:23.686477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:23.686443    2565 generic.go:356] "Generic (PLEG): container finished" podID="16b7329c-2031-4869-96fd-de225ca919a1" containerID="a8074a5f753ff72ad88198616a14b5d4538569cf18275ecf68587a145478d475" exitCode=0
Jul 06 19:52:23.686899 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:23.686541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerDied","Data":"a8074a5f753ff72ad88198616a14b5d4538569cf18275ecf68587a145478d475"}
Jul 06 19:52:31.718018 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:31.717987    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerStarted","Data":"a95155e6e29fb6630c88cc765cf7b3b587d25b85b51186ba793a05dce05d209b"}
Jul 06 19:52:31.739756 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:31.739700    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" podStartSLOduration=2.075698996 podStartE2EDuration="19.739679383s" totalImagesPullingTime="17.663980387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:31.738214307 +0000 UTC m=+250.000193175" watchObservedRunningTime="2026-07-06 19:52:31.739679383 +0000 UTC m=+250.001658248"
Jul 06 19:52:32.722803 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:32.722770    2565 generic.go:356] "Generic (PLEG): container finished" podID="16b7329c-2031-4869-96fd-de225ca919a1" containerID="a95155e6e29fb6630c88cc765cf7b3b587d25b85b51186ba793a05dce05d209b" exitCode=0
Jul 06 19:52:32.723192 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:32.722852    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerDied","Data":"a95155e6e29fb6630c88cc765cf7b3b587d25b85b51186ba793a05dce05d209b"}
Jul 06 19:52:33.886599 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.886568    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:33.967171 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.967139    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle\") pod \"16b7329c-2031-4869-96fd-de225ca919a1\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") "
Jul 06 19:52:33.967171 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.967176    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpxhn\" (UniqueName: \"kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn\") pod \"16b7329c-2031-4869-96fd-de225ca919a1\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") "
Jul 06 19:52:33.967348 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.967235    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util\") pod \"16b7329c-2031-4869-96fd-de225ca919a1\" (UID: \"16b7329c-2031-4869-96fd-de225ca919a1\") "
Jul 06 19:52:33.967787 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.967758    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle" (OuterVolumeSpecName: "bundle") pod "16b7329c-2031-4869-96fd-de225ca919a1" (UID: "16b7329c-2031-4869-96fd-de225ca919a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:52:33.969290 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.969261    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn" (OuterVolumeSpecName: "kube-api-access-cpxhn") pod "16b7329c-2031-4869-96fd-de225ca919a1" (UID: "16b7329c-2031-4869-96fd-de225ca919a1"). InnerVolumeSpecName "kube-api-access-cpxhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:52:33.972135 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:33.972115    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util" (OuterVolumeSpecName: "util") pod "16b7329c-2031-4869-96fd-de225ca919a1" (UID: "16b7329c-2031-4869-96fd-de225ca919a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:52:34.068639 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.068601    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:34.068639 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.068630    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpxhn\" (UniqueName: \"kubernetes.io/projected/16b7329c-2031-4869-96fd-de225ca919a1-kube-api-access-cpxhn\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:34.068639 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.068639    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b7329c-2031-4869-96fd-de225ca919a1-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:34.736065 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.736038    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw"
Jul 06 19:52:34.736227 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.736063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sx8dw" event={"ID":"16b7329c-2031-4869-96fd-de225ca919a1","Type":"ContainerDied","Data":"b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8"}
Jul 06 19:52:34.736227 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:34.736140    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2322c38e2f2aea2c6edfd2132a4911a4f54f9297d8ad252a539b807b38878f8"
Jul 06 19:52:40.128158 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128120    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"]
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128437    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="extract"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128455    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="extract"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128486    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="util"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128493    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="util"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128507    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="pull"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128515    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="pull"
Jul 06 19:52:40.128718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.128572    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b7329c-2031-4869-96fd-de225ca919a1" containerName="extract"
Jul 06 19:52:40.131491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.131470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.137548 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.137520    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 06 19:52:40.137657 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.137548    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5xd42\""
Jul 06 19:52:40.137657 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.137594    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:52:40.155948 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.155923    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"]
Jul 06 19:52:40.212409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.212383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7w4x\" (UniqueName: \"kubernetes.io/projected/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-kube-api-access-m7w4x\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.212509 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.212429    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.313693 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.313666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.313780 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.313708    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7w4x\" (UniqueName: \"kubernetes.io/projected/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-kube-api-access-m7w4x\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.314042 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.314025    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.325617 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.325597    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7w4x\" (UniqueName: \"kubernetes.io/projected/b25814a3-8a54-4088-8c59-b9ea36aa5b2c-kube-api-access-m7w4x\") pod \"cert-manager-operator-controller-manager-55c444bf86-wf92m\" (UID: \"b25814a3-8a54-4088-8c59-b9ea36aa5b2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.444228 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.444175    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"
Jul 06 19:52:40.574410 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.574387    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m"]
Jul 06 19:52:40.577195 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:52:40.577170    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25814a3_8a54_4088_8c59_b9ea36aa5b2c.slice/crio-0d107402b6c50bf4ab8777c87a15d84694062c72fbe09f7bf8ab8a64cdb1ffd0 WatchSource:0}: Error finding container 0d107402b6c50bf4ab8777c87a15d84694062c72fbe09f7bf8ab8a64cdb1ffd0: Status 404 returned error can't find the container with id 0d107402b6c50bf4ab8777c87a15d84694062c72fbe09f7bf8ab8a64cdb1ffd0
Jul 06 19:52:40.757168 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:40.757139    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m" event={"ID":"b25814a3-8a54-4088-8c59-b9ea36aa5b2c","Type":"ContainerStarted","Data":"0d107402b6c50bf4ab8777c87a15d84694062c72fbe09f7bf8ab8a64cdb1ffd0"}
Jul 06 19:52:42.766456 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:42.766422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m" event={"ID":"b25814a3-8a54-4088-8c59-b9ea36aa5b2c","Type":"ContainerStarted","Data":"a4a6b372cbf4260f6960ce70f0e626a58a2418ef2520425612081cf93c027af9"}
Jul 06 19:52:42.791869 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:42.791780    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wf92m" podStartSLOduration=0.833629648 podStartE2EDuration="2.791763655s" totalImagesPullingTime="1.958134007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:42.789927832 +0000 UTC m=+261.051906689" watchObservedRunningTime="2026-07-06 19:52:42.791763655 +0000 UTC m=+261.053742522"
Jul 06 19:52:43.652222 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.652188    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nmvgj"]
Jul 06 19:52:43.655138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.655118    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.657747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.657725    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rvxl7\""
Jul 06 19:52:43.658831 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.658818    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 19:52:43.658890 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.658825    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 19:52:43.667542 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.667520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nmvgj"]
Jul 06 19:52:43.739078 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.739051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl6dk\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-kube-api-access-dl6dk\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.739182 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.739086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.839614 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.839589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl6dk\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-kube-api-access-dl6dk\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.839901 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.839623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.849717 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.849692    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.849883 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.849865    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl6dk\" (UniqueName: \"kubernetes.io/projected/43866936-575d-4fc8-89db-930cbffbf22d-kube-api-access-dl6dk\") pod \"cert-manager-webhook-6486ffd997-nmvgj\" (UID: \"43866936-575d-4fc8-89db-930cbffbf22d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:43.963892 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:43.963803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:44.066714 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.066685    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"]
Jul 06 19:52:44.069980 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.069959    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.072489 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.072470    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8tsp2\""
Jul 06 19:52:44.072605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.072508    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:52:44.072605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.072508    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:52:44.083392 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.079766    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"]
Jul 06 19:52:44.094173 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.094141    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nmvgj"]
Jul 06 19:52:44.097056 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:52:44.097037    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43866936_575d_4fc8_89db_930cbffbf22d.slice/crio-0c21d30b72974d69dfa99710e825d014b7f57dd9dba07d1737b5272b1eb75979 WatchSource:0}: Error finding container 0c21d30b72974d69dfa99710e825d014b7f57dd9dba07d1737b5272b1eb75979: Status 404 returned error can't find the container with id 0c21d30b72974d69dfa99710e825d014b7f57dd9dba07d1737b5272b1eb75979
Jul 06 19:52:44.142640 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.142611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.142798 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.142680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.142798 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.142740    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5pn\" (UniqueName: \"kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.243779 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.243687    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.243779 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.243765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.243991 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.243801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5pn\" (UniqueName: \"kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.244102 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.244080    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.244165 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.244147    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.252944 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.252900    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5pn\" (UniqueName: \"kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.385613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.385581    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:44.517317 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.517286    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"]
Jul 06 19:52:44.520068 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:52:44.520039    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1501bb9b_d844_4ab2_86d1_5d0597236047.slice/crio-143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece WatchSource:0}: Error finding container 143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece: Status 404 returned error can't find the container with id 143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece
Jul 06 19:52:44.774852 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.774776    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj" event={"ID":"43866936-575d-4fc8-89db-930cbffbf22d","Type":"ContainerStarted","Data":"0c21d30b72974d69dfa99710e825d014b7f57dd9dba07d1737b5272b1eb75979"}
Jul 06 19:52:44.776086 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.776064    2565 generic.go:356] "Generic (PLEG): container finished" podID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerID="6694240d6f097623eadc1bdc38a4534b9e727ef9fa7f4e565b57b683cd3ba24c" exitCode=0
Jul 06 19:52:44.776152 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.776111    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h" event={"ID":"1501bb9b-d844-4ab2-86d1-5d0597236047","Type":"ContainerDied","Data":"6694240d6f097623eadc1bdc38a4534b9e727ef9fa7f4e565b57b683cd3ba24c"}
Jul 06 19:52:44.776152 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:44.776141    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h" event={"ID":"1501bb9b-d844-4ab2-86d1-5d0597236047","Type":"ContainerStarted","Data":"143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece"}
Jul 06 19:52:48.525285 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.525252    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"]
Jul 06 19:52:48.528382 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.528365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.530834 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.530805    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hhtd8\""
Jul 06 19:52:48.536376 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.536357    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"]
Jul 06 19:52:48.681463 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.681430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.681632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.681494    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5c6\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-kube-api-access-rg5c6\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.781953 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.781861    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5c6\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-kube-api-access-rg5c6\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.781953 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.781904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.789572 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.789540    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.789813 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.789788    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5c6\" (UniqueName: \"kubernetes.io/projected/21cc4ceb-48ca-4422-a8e1-ab10824e21eb-kube-api-access-rg5c6\") pod \"cert-manager-cainjector-559cdd8c87-br9v5\" (UID: \"21cc4ceb-48ca-4422-a8e1-ab10824e21eb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.795013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.794985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj" event={"ID":"43866936-575d-4fc8-89db-930cbffbf22d","Type":"ContainerStarted","Data":"41ef4151a393a7535980c27e589a649e99aafbe939714506513ce2ec8c3ba676"}
Jul 06 19:52:48.795125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.795051    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:52:48.797213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.797192    2565 generic.go:356] "Generic (PLEG): container finished" podID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerID="14ced887a0f845bfebbf3ed0d833bb9c7b0d873a6901398ebe9ca92f022a1577" exitCode=0
Jul 06 19:52:48.797312 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.797255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h" event={"ID":"1501bb9b-d844-4ab2-86d1-5d0597236047","Type":"ContainerDied","Data":"14ced887a0f845bfebbf3ed0d833bb9c7b0d873a6901398ebe9ca92f022a1577"}
Jul 06 19:52:48.811630 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.811591    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj" podStartSLOduration=2.212194198 podStartE2EDuration="5.811579537s" totalImagesPullingTime="3.599385339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:48.811064662 +0000 UTC m=+267.073043525" watchObservedRunningTime="2026-07-06 19:52:48.811579537 +0000 UTC m=+267.073558402"
Jul 06 19:52:48.837607 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.837587    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"
Jul 06 19:52:48.972368 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:48.972337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-br9v5"]
Jul 06 19:52:48.974581 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:52:48.974554    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21cc4ceb_48ca_4422_a8e1_ab10824e21eb.slice/crio-51f255fb79ad8a1d44054b4f8dc4808a43bb9617c0838ed4df5b47987491766d WatchSource:0}: Error finding container 51f255fb79ad8a1d44054b4f8dc4808a43bb9617c0838ed4df5b47987491766d: Status 404 returned error can't find the container with id 51f255fb79ad8a1d44054b4f8dc4808a43bb9617c0838ed4df5b47987491766d
Jul 06 19:52:49.803498 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:49.803467    2565 generic.go:356] "Generic (PLEG): container finished" podID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerID="ea7591b9959fe6938d40c3e000ab4a535f795146f91cf86aa1c8aca482ef95c8" exitCode=0
Jul 06 19:52:49.803878 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:49.803547    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h" event={"ID":"1501bb9b-d844-4ab2-86d1-5d0597236047","Type":"ContainerDied","Data":"ea7591b9959fe6938d40c3e000ab4a535f795146f91cf86aa1c8aca482ef95c8"}
Jul 06 19:52:49.805036 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:49.805009    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5" event={"ID":"21cc4ceb-48ca-4422-a8e1-ab10824e21eb","Type":"ContainerStarted","Data":"8bc1e2dfc75f1f7a2c4dd894da6ab1d0b2f4c562c1b7b7ba13089c3e19111963"}
Jul 06 19:52:49.805139 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:49.805049    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5" event={"ID":"21cc4ceb-48ca-4422-a8e1-ab10824e21eb","Type":"ContainerStarted","Data":"51f255fb79ad8a1d44054b4f8dc4808a43bb9617c0838ed4df5b47987491766d"}
Jul 06 19:52:49.837843 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:49.837795    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-br9v5" podStartSLOduration=1.837780194 podStartE2EDuration="1.837780194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:49.835978542 +0000 UTC m=+268.097957409" watchObservedRunningTime="2026-07-06 19:52:49.837780194 +0000 UTC m=+268.099759058"
Jul 06 19:52:50.936379 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:50.936356    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:51.000132 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.000098    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn5pn\" (UniqueName: \"kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn\") pod \"1501bb9b-d844-4ab2-86d1-5d0597236047\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") "
Jul 06 19:52:51.000305 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.000138    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle\") pod \"1501bb9b-d844-4ab2-86d1-5d0597236047\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") "
Jul 06 19:52:51.000305 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.000195    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util\") pod \"1501bb9b-d844-4ab2-86d1-5d0597236047\" (UID: \"1501bb9b-d844-4ab2-86d1-5d0597236047\") "
Jul 06 19:52:51.000566 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.000542    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle" (OuterVolumeSpecName: "bundle") pod "1501bb9b-d844-4ab2-86d1-5d0597236047" (UID: "1501bb9b-d844-4ab2-86d1-5d0597236047"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:52:51.002215 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.002191    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn" (OuterVolumeSpecName: "kube-api-access-nn5pn") pod "1501bb9b-d844-4ab2-86d1-5d0597236047" (UID: "1501bb9b-d844-4ab2-86d1-5d0597236047"). InnerVolumeSpecName "kube-api-access-nn5pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:52:51.004790 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.004767    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util" (OuterVolumeSpecName: "util") pod "1501bb9b-d844-4ab2-86d1-5d0597236047" (UID: "1501bb9b-d844-4ab2-86d1-5d0597236047"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:52:51.101322 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.101250    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn5pn\" (UniqueName: \"kubernetes.io/projected/1501bb9b-d844-4ab2-86d1-5d0597236047-kube-api-access-nn5pn\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:51.101322 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.101275    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:51.101322 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.101284    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1501bb9b-d844-4ab2-86d1-5d0597236047-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:52:51.814491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.814459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h" event={"ID":"1501bb9b-d844-4ab2-86d1-5d0597236047","Type":"ContainerDied","Data":"143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece"}
Jul 06 19:52:51.814491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.814480    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdvn7h"
Jul 06 19:52:51.814690 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:51.814505    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="143ef32b05d0b7fbb080c2319b02a97cd9e8db4b1870b6635f65b01863e54ece"
Jul 06 19:52:54.807434 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:52:54.807405    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nmvgj"
Jul 06 19:53:10.708766 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.708733    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"]
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.708979    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="util"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.708990    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="util"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.709000    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="extract"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.709006    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="extract"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.709019    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="pull"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.709024    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="pull"
Jul 06 19:53:10.709170 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.709071    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1501bb9b-d844-4ab2-86d1-5d0597236047" containerName="extract"
Jul 06 19:53:10.712996 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.712975    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.715380 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.715361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8tsp2\""
Jul 06 19:53:10.715490 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.715379    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:53:10.715490 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.715426    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:53:10.720866 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.720843    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"]
Jul 06 19:53:10.838564 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.838541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nljv7\" (UniqueName: \"kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.838677 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.838570    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.838677 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.838598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.939795 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.939772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.939931 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.939832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nljv7\" (UniqueName: \"kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.939931 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.939856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.940129 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.940112    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.940198 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.940179    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:10.948381 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:10.948357    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nljv7\" (UniqueName: \"kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:11.023088 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:11.023066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:11.152045 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:11.152019    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"]
Jul 06 19:53:11.154400 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:11.154377    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc75719f_7531_422c_9507_7f4c97678a5d.slice/crio-dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01 WatchSource:0}: Error finding container dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01: Status 404 returned error can't find the container with id dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01
Jul 06 19:53:11.890569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:11.890539    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc75719f-7531-422c-9507-7f4c97678a5d" containerID="8baabc2d0c5afb3fce000402e58f1083b37c13ffc158dbcf73056cb847ef7a91" exitCode=0
Jul 06 19:53:11.890999 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:11.890573    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz" event={"ID":"cc75719f-7531-422c-9507-7f4c97678a5d","Type":"ContainerDied","Data":"8baabc2d0c5afb3fce000402e58f1083b37c13ffc158dbcf73056cb847ef7a91"}
Jul 06 19:53:11.890999 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:11.890618    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz" event={"ID":"cc75719f-7531-422c-9507-7f4c97678a5d","Type":"ContainerStarted","Data":"dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01"}
Jul 06 19:53:12.896072 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:12.896035    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc75719f-7531-422c-9507-7f4c97678a5d" containerID="9250c484525bd9240980c0dd631c26ca73314059ffbc125748b87e35226f5e8c" exitCode=0
Jul 06 19:53:12.896480 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:12.896106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz" event={"ID":"cc75719f-7531-422c-9507-7f4c97678a5d","Type":"ContainerDied","Data":"9250c484525bd9240980c0dd631c26ca73314059ffbc125748b87e35226f5e8c"}
Jul 06 19:53:13.902029 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:13.901994    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc75719f-7531-422c-9507-7f4c97678a5d" containerID="ba84c73ef6ec84ca16193f5b79a89233b690205dc98be05ab4b01f24293e4b76" exitCode=0
Jul 06 19:53:13.902400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:13.902043    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz" event={"ID":"cc75719f-7531-422c-9507-7f4c97678a5d","Type":"ContainerDied","Data":"ba84c73ef6ec84ca16193f5b79a89233b690205dc98be05ab4b01f24293e4b76"}
Jul 06 19:53:15.026713 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.026689    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:15.169435 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.169349    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nljv7\" (UniqueName: \"kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7\") pod \"cc75719f-7531-422c-9507-7f4c97678a5d\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") "
Jul 06 19:53:15.169435 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.169392    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util\") pod \"cc75719f-7531-422c-9507-7f4c97678a5d\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") "
Jul 06 19:53:15.169435 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.169436    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle\") pod \"cc75719f-7531-422c-9507-7f4c97678a5d\" (UID: \"cc75719f-7531-422c-9507-7f4c97678a5d\") "
Jul 06 19:53:15.170200 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.170161    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle" (OuterVolumeSpecName: "bundle") pod "cc75719f-7531-422c-9507-7f4c97678a5d" (UID: "cc75719f-7531-422c-9507-7f4c97678a5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:53:15.171516 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.171493    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7" (OuterVolumeSpecName: "kube-api-access-nljv7") pod "cc75719f-7531-422c-9507-7f4c97678a5d" (UID: "cc75719f-7531-422c-9507-7f4c97678a5d"). InnerVolumeSpecName "kube-api-access-nljv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:53:15.175568 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.175532    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util" (OuterVolumeSpecName: "util") pod "cc75719f-7531-422c-9507-7f4c97678a5d" (UID: "cc75719f-7531-422c-9507-7f4c97678a5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:53:15.270473 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.270437    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nljv7\" (UniqueName: \"kubernetes.io/projected/cc75719f-7531-422c-9507-7f4c97678a5d-kube-api-access-nljv7\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:15.270473 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.270467    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:15.270473 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.270477    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc75719f-7531-422c-9507-7f4c97678a5d-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:15.911982 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.911954    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz"
Jul 06 19:53:15.912155 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.911961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsbmz" event={"ID":"cc75719f-7531-422c-9507-7f4c97678a5d","Type":"ContainerDied","Data":"dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01"}
Jul 06 19:53:15.912155 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:15.912071    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dac39703d18ef69bb3d87966e91d547b4edd5005b05a65467383908b2f9b7f01"
Jul 06 19:53:22.537805 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.537774    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:53:22.550503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.550476    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:53:22.551644 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.551618    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 19:53:22.709327 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709191    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"]
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709411    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="pull"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709422    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="pull"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709438    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="extract"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709443    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="extract"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709453    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="util"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709458    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="util"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.709497    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc75719f-7531-422c-9507-7f4c97678a5d" containerName="extract"
Jul 06 19:53:22.712900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.712842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.715661 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.715640    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6l89k\""
Jul 06 19:53:22.715661 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.715657    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 19:53:22.715832 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.715720    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 19:53:22.716079 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.716062    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 19:53:22.716349 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.716333    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 19:53:22.728323 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.728300    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"]
Jul 06 19:53:22.823953 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.823847    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.823953 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.823925    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glvg\" (UniqueName: \"kubernetes.io/projected/4e97320e-57bb-4143-bca7-1f180b3166e0-kube-api-access-5glvg\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.824138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.823978    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.824138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.824027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.824138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.824098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925144 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925329 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925329 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glvg\" (UniqueName: \"kubernetes.io/projected/4e97320e-57bb-4143-bca7-1f180b3166e0-kube-api-access-5glvg\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925329 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925329 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.925745 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.925711    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.927613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.927585    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e97320e-57bb-4143-bca7-1f180b3166e0-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.927613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.927610    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.927785 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.927667    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e97320e-57bb-4143-bca7-1f180b3166e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:22.933016 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:22.932990    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glvg\" (UniqueName: \"kubernetes.io/projected/4e97320e-57bb-4143-bca7-1f180b3166e0-kube-api-access-5glvg\") pod \"opendatahub-operator-controller-manager-74dff95f86-mlztn\" (UID: \"4e97320e-57bb-4143-bca7-1f180b3166e0\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:23.022115 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:23.022083    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:23.161016 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:23.160951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"]
Jul 06 19:53:23.163831 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:23.163808    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e97320e_57bb_4143_bca7_1f180b3166e0.slice/crio-82238b90fa13acb5c004d6aaf1d786627b4976d8b73603588d6ef693a0792ab9 WatchSource:0}: Error finding container 82238b90fa13acb5c004d6aaf1d786627b4976d8b73603588d6ef693a0792ab9: Status 404 returned error can't find the container with id 82238b90fa13acb5c004d6aaf1d786627b4976d8b73603588d6ef693a0792ab9
Jul 06 19:53:23.949794 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:23.949761    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn" event={"ID":"4e97320e-57bb-4143-bca7-1f180b3166e0","Type":"ContainerStarted","Data":"82238b90fa13acb5c004d6aaf1d786627b4976d8b73603588d6ef693a0792ab9"}
Jul 06 19:53:26.966354 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:26.966312    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn" event={"ID":"4e97320e-57bb-4143-bca7-1f180b3166e0","Type":"ContainerStarted","Data":"01c507368bec3ee86f15f4035924be105253de8ea884e4db0b1d5f84ab576567"}
Jul 06 19:53:27.972203 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:27.972172    2565 generic.go:356] "Generic (PLEG): container finished" podID="4e97320e-57bb-4143-bca7-1f180b3166e0" containerID="01c507368bec3ee86f15f4035924be105253de8ea884e4db0b1d5f84ab576567" exitCode=0
Jul 06 19:53:27.972580 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:27.972212    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn" event={"ID":"4e97320e-57bb-4143-bca7-1f180b3166e0","Type":"ContainerDied","Data":"01c507368bec3ee86f15f4035924be105253de8ea884e4db0b1d5f84ab576567"}
Jul 06 19:53:27.972691 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:27.972676    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:53:28.978341 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:28.978256    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn" event={"ID":"4e97320e-57bb-4143-bca7-1f180b3166e0","Type":"ContainerStarted","Data":"41e131e7a06404336de3f4a7cf3e7c7f0a201e02e69be7cbcc0c35f6058f29cb"}
Jul 06 19:53:28.978686 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:28.978362    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:29.000441 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:29.000401    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn" podStartSLOduration=2.208874678 podStartE2EDuration="7.00038682s" totalImagesPullingTime="3.72780424s" totalInitContainerRuntime="1.063707902s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:28.99885667 +0000 UTC m=+307.260835536" watchObservedRunningTime="2026-07-06 19:53:29.00038682 +0000 UTC m=+307.262365697"
Jul 06 19:53:39.985575 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:39.985545    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-mlztn"
Jul 06 19:53:42.557251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.557222    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"]
Jul 06 19:53:42.674262 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.674233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"]
Jul 06 19:53:42.674402 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.674338    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.678113 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.678092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8hm46\""
Jul 06 19:53:42.678264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.678092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 19:53:42.678264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.678092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 19:53:42.678264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.678120    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 19:53:42.678452 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.678147    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 19:53:42.754054 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.754029    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk42t\" (UniqueName: \"kubernetes.io/projected/26160051-315c-4b17-80e5-013261520b9d-kube-api-access-qk42t\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.754161 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.754066    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26160051-315c-4b17-80e5-013261520b9d-tls-certs\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.754161 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.754094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26160051-315c-4b17-80e5-013261520b9d-tmp\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.809941 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.809863    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nddfk"]
Jul 06 19:53:42.828238 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.828213    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nddfk"]
Jul 06 19:53:42.828359 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.828300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.830726 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.830708    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vdqjg\""
Jul 06 19:53:42.830900 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.830881    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 19:53:42.854416 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.854394    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.854518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.854432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk42t\" (UniqueName: \"kubernetes.io/projected/26160051-315c-4b17-80e5-013261520b9d-kube-api-access-qk42t\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.854518 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.854474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26160051-315c-4b17-80e5-013261520b9d-tls-certs\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.854624 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.854514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6kvx\" (UniqueName: \"kubernetes.io/projected/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-kube-api-access-d6kvx\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.854624 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.854547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26160051-315c-4b17-80e5-013261520b9d-tmp\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.856638 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.856614    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26160051-315c-4b17-80e5-013261520b9d-tmp\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.856827 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.856812    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26160051-315c-4b17-80e5-013261520b9d-tls-certs\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.867112 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.867092    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk42t\" (UniqueName: \"kubernetes.io/projected/26160051-315c-4b17-80e5-013261520b9d-kube-api-access-qk42t\") pod \"kube-auth-proxy-747cfbfd67-x2zzn\" (UID: \"26160051-315c-4b17-80e5-013261520b9d\") " pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:42.954947 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.954923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.955064 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.954978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6kvx\" (UniqueName: \"kubernetes.io/projected/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-kube-api-access-d6kvx\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.955103 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:53:42.955080    2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 19:53:42.955153 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:53:42.955143    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert podName:c09eefa8-7f7a-4733-bfd2-2b2d22c3225b nodeName:}" failed. No retries permitted until 2026-07-06 19:53:43.455129731 +0000 UTC m=+321.717108577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert") pod "odh-model-controller-7998c67dc4-nddfk" (UID: "c09eefa8-7f7a-4733-bfd2-2b2d22c3225b") : secret "odh-model-controller-webhook-cert" not found
Jul 06 19:53:42.965994 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.965973    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6kvx\" (UniqueName: \"kubernetes.io/projected/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-kube-api-access-d6kvx\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:42.984739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:42.984716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"
Jul 06 19:53:43.132711 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.132682    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn"]
Jul 06 19:53:43.135670 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:43.135635    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26160051_315c_4b17_80e5_013261520b9d.slice/crio-22a282aa6d65173efe4ee455236e387de0fb1c5ef50b4e29fecf09cfd8d41ebc WatchSource:0}: Error finding container 22a282aa6d65173efe4ee455236e387de0fb1c5ef50b4e29fecf09cfd8d41ebc: Status 404 returned error can't find the container with id 22a282aa6d65173efe4ee455236e387de0fb1c5ef50b4e29fecf09cfd8d41ebc
Jul 06 19:53:43.458400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.458307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:43.460813 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.460778    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09eefa8-7f7a-4733-bfd2-2b2d22c3225b-cert\") pod \"odh-model-controller-7998c67dc4-nddfk\" (UID: \"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b\") " pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:43.740931 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.740843    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:43.788716 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.788646    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vvf4r"]
Jul 06 19:53:43.791720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.791595    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.796681 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.796408    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 19:53:43.805824 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.805527    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vvf4r"]
Jul 06 19:53:43.862067 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.861890    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbwmf\" (UniqueName: \"kubernetes.io/projected/2334aa6f-f988-4811-842b-346ead0e42c3-kube-api-access-kbwmf\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.862067 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.861967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.893989 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.893962    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nddfk"]
Jul 06 19:53:43.896112 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:43.896085    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09eefa8_7f7a_4733_bfd2_2b2d22c3225b.slice/crio-5d7d57ec0f66e9a109d5bdab89367ba0f54ae9b0eecde466438611dbb44dd7c7 WatchSource:0}: Error finding container 5d7d57ec0f66e9a109d5bdab89367ba0f54ae9b0eecde466438611dbb44dd7c7: Status 404 returned error can't find the container with id 5d7d57ec0f66e9a109d5bdab89367ba0f54ae9b0eecde466438611dbb44dd7c7
Jul 06 19:53:43.963127 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.963094    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwmf\" (UniqueName: \"kubernetes.io/projected/2334aa6f-f988-4811-842b-346ead0e42c3-kube-api-access-kbwmf\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.963315 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.963166    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.963315 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:53:43.963281    2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 19:53:43.963435 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:53:43.963343    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs podName:2334aa6f-f988-4811-842b-346ead0e42c3 nodeName:}" failed. No retries permitted until 2026-07-06 19:53:44.463323356 +0000 UTC m=+322.725302207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs") pod "model-serving-api-54d89bd5fc-vvf4r" (UID: "2334aa6f-f988-4811-842b-346ead0e42c3") : secret "model-serving-api-tls" not found
Jul 06 19:53:43.976342 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.976314    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbwmf\" (UniqueName: \"kubernetes.io/projected/2334aa6f-f988-4811-842b-346ead0e42c3-kube-api-access-kbwmf\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:43.980769 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.980737    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:43.984647 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.984628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:43.987479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.986978    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 19:53:43.987479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.987023    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 19:53:43.987479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.987108    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:53:43.987479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.987159    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w82m2\""
Jul 06 19:53:43.987479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.987207    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 19:53:43.994808 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:43.994776    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:44.044878 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.044847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn" event={"ID":"26160051-315c-4b17-80e5-013261520b9d","Type":"ContainerStarted","Data":"22a282aa6d65173efe4ee455236e387de0fb1c5ef50b4e29fecf09cfd8d41ebc"}
Jul 06 19:53:44.046266 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.046219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk" event={"ID":"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b","Type":"ContainerStarted","Data":"5d7d57ec0f66e9a109d5bdab89367ba0f54ae9b0eecde466438611dbb44dd7c7"}
Jul 06 19:53:44.063718 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063692    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.063842 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063733    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.063842 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063769    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.063842 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063797    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.063842 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063827    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plvpx\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.064038 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063867    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.064038 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.063953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165260 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165386 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165386 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165519 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165575 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165627 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165679 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plvpx\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.165930 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.165887    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.168055 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.168029    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.168203 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.168186    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.168401 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.168382    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.168467 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.168382    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.180368 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.180345    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.180690 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.180663    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plvpx\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx\") pod \"istiod-openshift-gateway-7574459656-6bn26\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.296544 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.296516    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:44.427339 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.427313    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:44.430387 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:44.430358    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4584ec65_2334_4977_b5ef_cd8f7d62428c.slice/crio-082be6281049339ba12be42dea313c0d114ae8849830269f0723a11fbc789216 WatchSource:0}: Error finding container 082be6281049339ba12be42dea313c0d114ae8849830269f0723a11fbc789216: Status 404 returned error can't find the container with id 082be6281049339ba12be42dea313c0d114ae8849830269f0723a11fbc789216
Jul 06 19:53:44.467071 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.467045    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:44.469176 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.469152    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2334aa6f-f988-4811-842b-346ead0e42c3-tls-certs\") pod \"model-serving-api-54d89bd5fc-vvf4r\" (UID: \"2334aa6f-f988-4811-842b-346ead0e42c3\") " pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:44.709200 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.709133    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:44.846624 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:44.846546    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vvf4r"]
Jul 06 19:53:44.850568 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:53:44.850541    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2334aa6f_f988_4811_842b_346ead0e42c3.slice/crio-48b142e106f680dc3d5cb7d9623343ff66fd787840f845d41bd4138c66e42882 WatchSource:0}: Error finding container 48b142e106f680dc3d5cb7d9623343ff66fd787840f845d41bd4138c66e42882: Status 404 returned error can't find the container with id 48b142e106f680dc3d5cb7d9623343ff66fd787840f845d41bd4138c66e42882
Jul 06 19:53:45.054561 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:45.054479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" event={"ID":"4584ec65-2334-4977-b5ef-cd8f7d62428c","Type":"ContainerStarted","Data":"082be6281049339ba12be42dea313c0d114ae8849830269f0723a11fbc789216"}
Jul 06 19:53:45.057721 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:45.057655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r" event={"ID":"2334aa6f-f988-4811-842b-346ead0e42c3","Type":"ContainerStarted","Data":"48b142e106f680dc3d5cb7d9623343ff66fd787840f845d41bd4138c66e42882"}
Jul 06 19:53:49.079982 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.079932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk" event={"ID":"c09eefa8-7f7a-4733-bfd2-2b2d22c3225b","Type":"ContainerStarted","Data":"b7f63ecc6ee5c95fa67d2cfea9858d7f5be7f56662ade6928e66e63f93601d50"}
Jul 06 19:53:49.080513 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.079994    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:53:49.081425 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.081392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn" event={"ID":"26160051-315c-4b17-80e5-013261520b9d","Type":"ContainerStarted","Data":"fb6cef1c551f86e7e62d3026a3db2a0bce0cc031ed4cba18c3c1caee3709c37a"}
Jul 06 19:53:49.082739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.082696    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r" event={"ID":"2334aa6f-f988-4811-842b-346ead0e42c3","Type":"ContainerStarted","Data":"f39c53d9ce663376522feb318eb7b3dfcbb29f9c7a9e02236637fb4ca18c65d6"}
Jul 06 19:53:49.082857 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.082804    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:53:49.097893 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.097844    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk" podStartSLOduration=2.8842687849999997 podStartE2EDuration="7.097831312s" totalImagesPullingTime="4.213562527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:49.096388786 +0000 UTC m=+327.358367650" watchObservedRunningTime="2026-07-06 19:53:49.097831312 +0000 UTC m=+327.359810226"
Jul 06 19:53:49.116503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.116463    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-747cfbfd67-x2zzn" podStartSLOduration=2.152629673 podStartE2EDuration="7.116451049s" totalImagesPullingTime="4.963821376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:49.114664682 +0000 UTC m=+327.376643545" watchObservedRunningTime="2026-07-06 19:53:49.116451049 +0000 UTC m=+327.378429912"
Jul 06 19:53:49.139326 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:49.139275    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r" podStartSLOduration=2.89152044 podStartE2EDuration="6.139260547s" totalImagesPullingTime="3.247740107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:49.139015381 +0000 UTC m=+327.400994245" watchObservedRunningTime="2026-07-06 19:53:49.139260547 +0000 UTC m=+327.401239411"
Jul 06 19:53:54.886020 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:54.885987    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:55.843281 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:55.843243    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:53:55.843351 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:55.843320    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:53:56.114943 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.114834    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" event={"ID":"4584ec65-2334-4977-b5ef-cd8f7d62428c","Type":"ContainerStarted","Data":"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"}
Jul 06 19:53:56.115388 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.114946    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerName="discovery" containerID="cri-o://4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11" gracePeriod=30
Jul 06 19:53:56.115388 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.115094    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:56.116986 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.116957    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-6bn26 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 19:53:56.117100 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.117010    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:53:56.140400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.140351    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" podStartSLOduration=1.730172388 podStartE2EDuration="13.140335476s" totalImagesPullingTime="11.410163088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:56.13872412 +0000 UTC m=+334.400702985" watchObservedRunningTime="2026-07-06 19:53:56.140335476 +0000 UTC m=+334.402314339"
Jul 06 19:53:56.354461 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.354435    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:56.461052 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.460959    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461052 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461014    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461052 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461050    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461286 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461086    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461286 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461133    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461286 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461186    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461286 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plvpx\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx\") pod \"4584ec65-2334-4977-b5ef-cd8f7d62428c\" (UID: \"4584ec65-2334-4977-b5ef-cd8f7d62428c\") "
Jul 06 19:53:56.461545 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461481    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:53:56.461783 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.461761    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-ca-configmap\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.463473 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463443    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts" (OuterVolumeSpecName: "cacerts") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:53:56.463613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463527    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:53:56.463712 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463691    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs" (OuterVolumeSpecName: "local-certs") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:53:56.463811 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463790    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx" (OuterVolumeSpecName: "kube-api-access-plvpx") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "kube-api-access-plvpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:53:56.463848 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463826    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:53:56.463848 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.463830    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4584ec65-2334-4977-b5ef-cd8f7d62428c" (UID: "4584ec65-2334-4977-b5ef-cd8f7d62428c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:53:56.562140 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562108    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-csr-dns-cert\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.562140 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562137    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plvpx\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-kube-api-access-plvpx\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.562140 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562149    2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-cacerts\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.562140 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562157    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-token\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.562405 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562167    2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4584ec65-2334-4977-b5ef-cd8f7d62428c-local-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:56.562405 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:56.562175    2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4584ec65-2334-4977-b5ef-cd8f7d62428c-istio-kubeconfig\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:53:57.120000 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.119966    2565 generic.go:356] "Generic (PLEG): container finished" podID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerID="4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11" exitCode=2
Jul 06 19:53:57.120400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.120012    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"
Jul 06 19:53:57.120400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.120056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" event={"ID":"4584ec65-2334-4977-b5ef-cd8f7d62428c","Type":"ContainerDied","Data":"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"}
Jul 06 19:53:57.120400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.120110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6bn26" event={"ID":"4584ec65-2334-4977-b5ef-cd8f7d62428c","Type":"ContainerDied","Data":"082be6281049339ba12be42dea313c0d114ae8849830269f0723a11fbc789216"}
Jul 06 19:53:57.120400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.120131    2565 scope.go:117] "RemoveContainer" containerID="4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"
Jul 06 19:53:57.128237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.128221    2565 scope.go:117] "RemoveContainer" containerID="4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"
Jul 06 19:53:57.128487 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:53:57.128469    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11\": container with ID starting with 4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11 not found: ID does not exist" containerID="4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"
Jul 06 19:53:57.128545 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.128496    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11"} err="failed to get container status \"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11\": rpc error: code = NotFound desc = could not find container \"4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11\": container with ID starting with 4c49c8788c59714f4bc2214d40413a254c5f3a18b0b445d82a843acb81143a11 not found: ID does not exist"
Jul 06 19:53:57.137890 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.137868    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:57.142330 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:57.142309    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6bn26"]
Jul 06 19:53:58.657846 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:53:58.657808    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" path="/var/lib/kubelet/pods/4584ec65-2334-4977-b5ef-cd8f7d62428c/volumes"
Jul 06 19:54:00.088992 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:00.088966    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nddfk"
Jul 06 19:54:00.090877 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:00.090856    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vvf4r"
Jul 06 19:54:43.689290 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.689253    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:43.689781 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.689655    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerName="discovery"
Jul 06 19:54:43.689781 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.689672    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerName="discovery"
Jul 06 19:54:43.689781 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.689749    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4584ec65-2334-4977-b5ef-cd8f7d62428c" containerName="discovery"
Jul 06 19:54:43.703340 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.703309    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:43.704131 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.704098    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:43.706033 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.706010    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 19:54:43.707199 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.707179    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 19:54:43.707411 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.707283    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5pt4c\""
Jul 06 19:54:43.817924 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.817870    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ws4h\" (UniqueName: \"kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h\") pod \"kuadrant-operator-catalog-257dl\" (UID: \"db31ce70-ffef-42dd-8bc6-1e00f7e34c93\") " pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:43.918984 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.918949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ws4h\" (UniqueName: \"kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h\") pod \"kuadrant-operator-catalog-257dl\" (UID: \"db31ce70-ffef-42dd-8bc6-1e00f7e34c93\") " pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:43.927781 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:43.927762    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ws4h\" (UniqueName: \"kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h\") pod \"kuadrant-operator-catalog-257dl\" (UID: \"db31ce70-ffef-42dd-8bc6-1e00f7e34c93\") " pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:44.014725 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.014698    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:44.040072 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.040048    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:44.162646 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.162614    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:44.165302 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:54:44.165277    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb31ce70_ffef_42dd_8bc6_1e00f7e34c93.slice/crio-af0a7ce9e5770d132bc2ecac9dc8d21c4018b35226cad5aaf702e4ce629ff8fa WatchSource:0}: Error finding container af0a7ce9e5770d132bc2ecac9dc8d21c4018b35226cad5aaf702e4ce629ff8fa: Status 404 returned error can't find the container with id af0a7ce9e5770d132bc2ecac9dc8d21c4018b35226cad5aaf702e4ce629ff8fa
Jul 06 19:54:44.249932 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.249896    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ll29l"]
Jul 06 19:54:44.254503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.254487    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:44.262363 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.262344    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ll29l"]
Jul 06 19:54:44.326564 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.326502    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-257dl" event={"ID":"db31ce70-ffef-42dd-8bc6-1e00f7e34c93","Type":"ContainerStarted","Data":"af0a7ce9e5770d132bc2ecac9dc8d21c4018b35226cad5aaf702e4ce629ff8fa"}
Jul 06 19:54:44.422610 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.422581    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hdbj\" (UniqueName: \"kubernetes.io/projected/4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0-kube-api-access-8hdbj\") pod \"kuadrant-operator-catalog-ll29l\" (UID: \"4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0\") " pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:44.523128 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.523103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hdbj\" (UniqueName: \"kubernetes.io/projected/4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0-kube-api-access-8hdbj\") pod \"kuadrant-operator-catalog-ll29l\" (UID: \"4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0\") " pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:44.531985 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.531957    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hdbj\" (UniqueName: \"kubernetes.io/projected/4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0-kube-api-access-8hdbj\") pod \"kuadrant-operator-catalog-ll29l\" (UID: \"4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0\") " pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:44.563879 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.563856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:44.693898 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:44.693874    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ll29l"]
Jul 06 19:54:44.740010 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:54:44.739978    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d2b2f55_7b56_44a6_86c7_00a82bb8c8f0.slice/crio-4443b7a4055b8a9969916dc6c635279c89391a5ee7a09cc6e80125348511dd16 WatchSource:0}: Error finding container 4443b7a4055b8a9969916dc6c635279c89391a5ee7a09cc6e80125348511dd16: Status 404 returned error can't find the container with id 4443b7a4055b8a9969916dc6c635279c89391a5ee7a09cc6e80125348511dd16
Jul 06 19:54:45.334272 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:45.334227    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ll29l" event={"ID":"4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0","Type":"ContainerStarted","Data":"4443b7a4055b8a9969916dc6c635279c89391a5ee7a09cc6e80125348511dd16"}
Jul 06 19:54:47.344382 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.344345    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ll29l" event={"ID":"4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0","Type":"ContainerStarted","Data":"f599ac145f9c257cd0070d51df0e0e52aecdbaef2360b5104497548184471bb7"}
Jul 06 19:54:47.345907 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.345878    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-257dl" event={"ID":"db31ce70-ffef-42dd-8bc6-1e00f7e34c93","Type":"ContainerStarted","Data":"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"}
Jul 06 19:54:47.346028 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.345982    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-257dl" podUID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" containerName="registry-server" containerID="cri-o://9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc" gracePeriod=2
Jul 06 19:54:47.361188 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.361147    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ll29l" podStartSLOduration=1.782536737 podStartE2EDuration="3.361130875s" totalImagesPullingTime="1.578594138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:47.359659276 +0000 UTC m=+385.621638140" watchObservedRunningTime="2026-07-06 19:54:47.361130875 +0000 UTC m=+385.623109739"
Jul 06 19:54:47.387410 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.387361    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-257dl" podStartSLOduration=2.237339739 podStartE2EDuration="4.38734475s" totalImagesPullingTime="2.150005011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:47.38626465 +0000 UTC m=+385.648243505" watchObservedRunningTime="2026-07-06 19:54:47.38734475 +0000 UTC m=+385.649323613"
Jul 06 19:54:47.583541 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.583516    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:47.748460 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.748374    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ws4h\" (UniqueName: \"kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h\") pod \"db31ce70-ffef-42dd-8bc6-1e00f7e34c93\" (UID: \"db31ce70-ffef-42dd-8bc6-1e00f7e34c93\") "
Jul 06 19:54:47.750510 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.750479    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h" (OuterVolumeSpecName: "kube-api-access-4ws4h") pod "db31ce70-ffef-42dd-8bc6-1e00f7e34c93" (UID: "db31ce70-ffef-42dd-8bc6-1e00f7e34c93"). InnerVolumeSpecName "kube-api-access-4ws4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:54:47.849666 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:47.849633    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ws4h\" (UniqueName: \"kubernetes.io/projected/db31ce70-ffef-42dd-8bc6-1e00f7e34c93-kube-api-access-4ws4h\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:54:48.351776 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.351745    2565 generic.go:356] "Generic (PLEG): container finished" podID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" containerID="9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc" exitCode=0
Jul 06 19:54:48.352196 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.351792    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-257dl"
Jul 06 19:54:48.352196 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.351833    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-257dl" event={"ID":"db31ce70-ffef-42dd-8bc6-1e00f7e34c93","Type":"ContainerDied","Data":"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"}
Jul 06 19:54:48.352196 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.351880    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-257dl" event={"ID":"db31ce70-ffef-42dd-8bc6-1e00f7e34c93","Type":"ContainerDied","Data":"af0a7ce9e5770d132bc2ecac9dc8d21c4018b35226cad5aaf702e4ce629ff8fa"}
Jul 06 19:54:48.352196 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.351906    2565 scope.go:117] "RemoveContainer" containerID="9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"
Jul 06 19:54:48.360996 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.360980    2565 scope.go:117] "RemoveContainer" containerID="9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"
Jul 06 19:54:48.361219 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:54:48.361201    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc\": container with ID starting with 9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc not found: ID does not exist" containerID="9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"
Jul 06 19:54:48.361259 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.361229    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc"} err="failed to get container status \"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc\": rpc error: code = NotFound desc = could not find container \"9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc\": container with ID starting with 9cd80212c8f5070948cdb772cc27e10fddea360d803fcb241eaf1489303b90bc not found: ID does not exist"
Jul 06 19:54:48.372327 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.372303    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:48.376501 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.376483    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-257dl"]
Jul 06 19:54:48.654651 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:48.654577    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" path="/var/lib/kubelet/pods/db31ce70-ffef-42dd-8bc6-1e00f7e34c93/volumes"
Jul 06 19:54:54.564993 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:54.564945    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:54.565511 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:54.565024    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:54.587130 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:54.587101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:55.405561 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:55.405533    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ll29l"
Jul 06 19:54:59.287861 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.287833    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"]
Jul 06 19:54:59.288264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.288135    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" containerName="registry-server"
Jul 06 19:54:59.288264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.288147    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" containerName="registry-server"
Jul 06 19:54:59.288264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.288194    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="db31ce70-ffef-42dd-8bc6-1e00f7e34c93" containerName="registry-server"
Jul 06 19:54:59.292608 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.292584    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.294832 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.294809    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pbbgj\""
Jul 06 19:54:59.301776 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.301753    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"]
Jul 06 19:54:59.432436 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.432412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.432576 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.432450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.432576 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.432528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzfdq\" (UniqueName: \"kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.533656 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.533632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.533757 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.533669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.533815 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.533793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdq\" (UniqueName: \"kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.534008 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.533990    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.534085 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.534035    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.549479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.549419    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzfdq\" (UniqueName: \"kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.602792 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.602771    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:54:59.735824 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.735793    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"]
Jul 06 19:54:59.739903 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:54:59.739878    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64231890_1a99_4aa1_8c56_5325e0261d05.slice/crio-0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d WatchSource:0}: Error finding container 0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d: Status 404 returned error can't find the container with id 0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d
Jul 06 19:54:59.740423 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.740397    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"]
Jul 06 19:54:59.740502 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.740489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:54:59.755289 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.755265    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"]
Jul 06 19:54:59.936392 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.936356    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:54:59.936540 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.936398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:54:59.936540 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:54:59.936482    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7tw5\" (UniqueName: \"kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.037119 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.037085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.037288 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.037129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.037288 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.037167    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7tw5\" (UniqueName: \"kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.037505 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.037484    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.037562 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.037543    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.044567 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.044542    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7tw5\" (UniqueName: \"kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.054378 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.054333    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:00.084386 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.084363    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"]
Jul 06 19:55:00.088776 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.088754    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.099499 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.099474    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"]
Jul 06 19:55:00.191160 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.191135    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"]
Jul 06 19:55:00.193153 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:00.193121    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0cd019_292c_4c35_8069_318eeea30798.slice/crio-62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803 WatchSource:0}: Error finding container 62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803: Status 404 returned error can't find the container with id 62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803
Jul 06 19:55:00.238942 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.238896    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlp2g\" (UniqueName: \"kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.239038 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.238956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.239038 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.238983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.340375 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.340312    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlp2g\" (UniqueName: \"kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.340375 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.340343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.340375 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.340369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.340741 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.340671    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.340741 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.340699    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.348304 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.348282    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlp2g\" (UniqueName: \"kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.399096 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.399070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:00.408259 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.408235    2565 generic.go:356] "Generic (PLEG): container finished" podID="64231890-1a99-4aa1-8c56-5325e0261d05" containerID="7ee68f22b247f8eeb9956ce6a44f737b6f98b54b3a8044cd4f879eadf1173617" exitCode=0
Jul 06 19:55:00.408353 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.408296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq" event={"ID":"64231890-1a99-4aa1-8c56-5325e0261d05","Type":"ContainerDied","Data":"7ee68f22b247f8eeb9956ce6a44f737b6f98b54b3a8044cd4f879eadf1173617"}
Jul 06 19:55:00.408401 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.408352    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq" event={"ID":"64231890-1a99-4aa1-8c56-5325e0261d05","Type":"ContainerStarted","Data":"0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d"}
Jul 06 19:55:00.410021 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.409890    2565 generic.go:356] "Generic (PLEG): container finished" podID="0a0cd019-292c-4c35-8069-318eeea30798" containerID="2144541af5e8fe017788a3367260e9d34fde9b7a1f7e62b4caafbef9b432b6c9" exitCode=0
Jul 06 19:55:00.410021 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.409966    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn" event={"ID":"0a0cd019-292c-4c35-8069-318eeea30798","Type":"ContainerDied","Data":"2144541af5e8fe017788a3367260e9d34fde9b7a1f7e62b4caafbef9b432b6c9"}
Jul 06 19:55:00.410021 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.409999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn" event={"ID":"0a0cd019-292c-4c35-8069-318eeea30798","Type":"ContainerStarted","Data":"62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803"}
Jul 06 19:55:00.490700 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.490670    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"]
Jul 06 19:55:00.495317 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.495296    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.504612 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.504588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"]
Jul 06 19:55:00.530591 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.530568    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"]
Jul 06 19:55:00.532469 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:00.532447    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325616cd_18f5_448e_bd27_75f394567b90.slice/crio-49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff WatchSource:0}: Error finding container 49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff: Status 404 returned error can't find the container with id 49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff
Jul 06 19:55:00.643233 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.643207    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.643330 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.643250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.643330 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.643283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rhbv\" (UniqueName: \"kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.744055 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.744028    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.744162 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.744067    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.744162 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.744089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rhbv\" (UniqueName: \"kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.744385 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.744366    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.744425 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.744410    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.752738 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.752711    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rhbv\" (UniqueName: \"kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.805675 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.805651    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:00.931761 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:00.931740    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"]
Jul 06 19:55:00.934307 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:00.934277    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831f918f_f685_4667_b76e_1ea3f5af7a91.slice/crio-9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883 WatchSource:0}: Error finding container 9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883: Status 404 returned error can't find the container with id 9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883
Jul 06 19:55:01.416538 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.416509    2565 generic.go:356] "Generic (PLEG): container finished" podID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerID="597780cbe6aab313fe2ba44fd262610858603bdc45fbcc39276a1aa4c9926c2b" exitCode=0
Jul 06 19:55:01.416863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.416593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm" event={"ID":"831f918f-f685-4667-b76e-1ea3f5af7a91","Type":"ContainerDied","Data":"597780cbe6aab313fe2ba44fd262610858603bdc45fbcc39276a1aa4c9926c2b"}
Jul 06 19:55:01.416863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.416642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm" event={"ID":"831f918f-f685-4667-b76e-1ea3f5af7a91","Type":"ContainerStarted","Data":"9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883"}
Jul 06 19:55:01.418360 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.418337    2565 generic.go:356] "Generic (PLEG): container finished" podID="64231890-1a99-4aa1-8c56-5325e0261d05" containerID="23ee246303b7cdd19d5ad3db4e88cad4ed10e346ac16fe214d0e3d46a06edbfb" exitCode=0
Jul 06 19:55:01.418465 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.418417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq" event={"ID":"64231890-1a99-4aa1-8c56-5325e0261d05","Type":"ContainerDied","Data":"23ee246303b7cdd19d5ad3db4e88cad4ed10e346ac16fe214d0e3d46a06edbfb"}
Jul 06 19:55:01.419858 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.419839    2565 generic.go:356] "Generic (PLEG): container finished" podID="325616cd-18f5-448e-bd27-75f394567b90" containerID="19fd4fb642dbbddf373de0dcbfde556ec12ef45840d8a3fee44c096b738ec6ec" exitCode=0
Jul 06 19:55:01.419944 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.419899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk" event={"ID":"325616cd-18f5-448e-bd27-75f394567b90","Type":"ContainerDied","Data":"19fd4fb642dbbddf373de0dcbfde556ec12ef45840d8a3fee44c096b738ec6ec"}
Jul 06 19:55:01.419989 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:01.419951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk" event={"ID":"325616cd-18f5-448e-bd27-75f394567b90","Type":"ContainerStarted","Data":"49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff"}
Jul 06 19:55:02.428000 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.427969    2565 generic.go:356] "Generic (PLEG): container finished" podID="64231890-1a99-4aa1-8c56-5325e0261d05" containerID="50a1c7f70b2b75ebf34dad6d9b03cde113128bc2c44f30c2e6e759f27e488017" exitCode=0
Jul 06 19:55:02.428414 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.428031    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq" event={"ID":"64231890-1a99-4aa1-8c56-5325e0261d05","Type":"ContainerDied","Data":"50a1c7f70b2b75ebf34dad6d9b03cde113128bc2c44f30c2e6e759f27e488017"}
Jul 06 19:55:02.430278 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.430257    2565 generic.go:356] "Generic (PLEG): container finished" podID="325616cd-18f5-448e-bd27-75f394567b90" containerID="3bc7c3abab0c206fb456500492f8662a63a23171496e8b67f2afe166dd977e15" exitCode=0
Jul 06 19:55:02.430378 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.430344    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk" event={"ID":"325616cd-18f5-448e-bd27-75f394567b90","Type":"ContainerDied","Data":"3bc7c3abab0c206fb456500492f8662a63a23171496e8b67f2afe166dd977e15"}
Jul 06 19:55:02.432259 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.432238    2565 generic.go:356] "Generic (PLEG): container finished" podID="0a0cd019-292c-4c35-8069-318eeea30798" containerID="60641a060163e27707c5d0e4b651599a51756375c18e442336cc2fffc9622594" exitCode=0
Jul 06 19:55:02.432352 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:02.432314    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn" event={"ID":"0a0cd019-292c-4c35-8069-318eeea30798","Type":"ContainerDied","Data":"60641a060163e27707c5d0e4b651599a51756375c18e442336cc2fffc9622594"}
Jul 06 19:55:03.438725 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.438693    2565 generic.go:356] "Generic (PLEG): container finished" podID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerID="a6d504d8112830a5285f6df2facd16306651c6ebd517f1a012fc734cf2f6126e" exitCode=0
Jul 06 19:55:03.439165 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.438766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm" event={"ID":"831f918f-f685-4667-b76e-1ea3f5af7a91","Type":"ContainerDied","Data":"a6d504d8112830a5285f6df2facd16306651c6ebd517f1a012fc734cf2f6126e"}
Jul 06 19:55:03.441229 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.441206    2565 generic.go:356] "Generic (PLEG): container finished" podID="325616cd-18f5-448e-bd27-75f394567b90" containerID="4a65eb45c28a4f5f1bf868f18b119e63f6e81715da5d9380439547d68a11e8d5" exitCode=0
Jul 06 19:55:03.441333 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.441314    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk" event={"ID":"325616cd-18f5-448e-bd27-75f394567b90","Type":"ContainerDied","Data":"4a65eb45c28a4f5f1bf868f18b119e63f6e81715da5d9380439547d68a11e8d5"}
Jul 06 19:55:03.443752 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.443729    2565 generic.go:356] "Generic (PLEG): container finished" podID="0a0cd019-292c-4c35-8069-318eeea30798" containerID="f22c8c6b124a4f4656e1fbf8ce3a1a8ca104654d1d4b6e36f0ec44315f81f6d3" exitCode=0
Jul 06 19:55:03.443752 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.443753    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn" event={"ID":"0a0cd019-292c-4c35-8069-318eeea30798","Type":"ContainerDied","Data":"f22c8c6b124a4f4656e1fbf8ce3a1a8ca104654d1d4b6e36f0ec44315f81f6d3"}
Jul 06 19:55:03.632734 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.632707    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:55:03.665620 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.665595    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util\") pod \"64231890-1a99-4aa1-8c56-5325e0261d05\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") "
Jul 06 19:55:03.665739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.665666    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzfdq\" (UniqueName: \"kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq\") pod \"64231890-1a99-4aa1-8c56-5325e0261d05\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") "
Jul 06 19:55:03.665739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.665698    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle\") pod \"64231890-1a99-4aa1-8c56-5325e0261d05\" (UID: \"64231890-1a99-4aa1-8c56-5325e0261d05\") "
Jul 06 19:55:03.666196 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.666164    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle" (OuterVolumeSpecName: "bundle") pod "64231890-1a99-4aa1-8c56-5325e0261d05" (UID: "64231890-1a99-4aa1-8c56-5325e0261d05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:03.667769 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.667741    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq" (OuterVolumeSpecName: "kube-api-access-kzfdq") pod "64231890-1a99-4aa1-8c56-5325e0261d05" (UID: "64231890-1a99-4aa1-8c56-5325e0261d05"). InnerVolumeSpecName "kube-api-access-kzfdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:03.674418 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.674389    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util" (OuterVolumeSpecName: "util") pod "64231890-1a99-4aa1-8c56-5325e0261d05" (UID: "64231890-1a99-4aa1-8c56-5325e0261d05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:03.766340 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.766313    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzfdq\" (UniqueName: \"kubernetes.io/projected/64231890-1a99-4aa1-8c56-5325e0261d05-kube-api-access-kzfdq\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:03.766340 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.766339    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:03.766510 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:03.766347    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64231890-1a99-4aa1-8c56-5325e0261d05-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.450446 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.450417    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq"
Jul 06 19:55:04.450446 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.450443    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq" event={"ID":"64231890-1a99-4aa1-8c56-5325e0261d05","Type":"ContainerDied","Data":"0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d"}
Jul 06 19:55:04.450930 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.450486    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0204c1a0c5ca036ccf6bee5d79677a73c75fc17b1d9bf0658b69ad9f80e9172d"
Jul 06 19:55:04.452613 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.452592    2565 generic.go:356] "Generic (PLEG): container finished" podID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerID="68b04db4332f41dd3111ff9ef728d1a59af6b4b61a56868f8ceefe82343a9e26" exitCode=0
Jul 06 19:55:04.452711 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.452653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm" event={"ID":"831f918f-f685-4667-b76e-1ea3f5af7a91","Type":"ContainerDied","Data":"68b04db4332f41dd3111ff9ef728d1a59af6b4b61a56868f8ceefe82343a9e26"}
Jul 06 19:55:04.591100 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.591074    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:04.618884 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.618855    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:04.671721 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671691    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle\") pod \"0a0cd019-292c-4c35-8069-318eeea30798\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") "
Jul 06 19:55:04.671872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671774    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlp2g\" (UniqueName: \"kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g\") pod \"325616cd-18f5-448e-bd27-75f394567b90\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") "
Jul 06 19:55:04.671872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671808    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle\") pod \"325616cd-18f5-448e-bd27-75f394567b90\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") "
Jul 06 19:55:04.672014 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671870    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7tw5\" (UniqueName: \"kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5\") pod \"0a0cd019-292c-4c35-8069-318eeea30798\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") "
Jul 06 19:55:04.672014 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671899    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util\") pod \"325616cd-18f5-448e-bd27-75f394567b90\" (UID: \"325616cd-18f5-448e-bd27-75f394567b90\") "
Jul 06 19:55:04.672014 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.671993    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util\") pod \"0a0cd019-292c-4c35-8069-318eeea30798\" (UID: \"0a0cd019-292c-4c35-8069-318eeea30798\") "
Jul 06 19:55:04.672841 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.672806    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle" (OuterVolumeSpecName: "bundle") pod "0a0cd019-292c-4c35-8069-318eeea30798" (UID: "0a0cd019-292c-4c35-8069-318eeea30798"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:04.672995 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.672680    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle" (OuterVolumeSpecName: "bundle") pod "325616cd-18f5-448e-bd27-75f394567b90" (UID: "325616cd-18f5-448e-bd27-75f394567b90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:04.674704 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.674669    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5" (OuterVolumeSpecName: "kube-api-access-v7tw5") pod "0a0cd019-292c-4c35-8069-318eeea30798" (UID: "0a0cd019-292c-4c35-8069-318eeea30798"). InnerVolumeSpecName "kube-api-access-v7tw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:04.676247 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.676219    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g" (OuterVolumeSpecName: "kube-api-access-hlp2g") pod "325616cd-18f5-448e-bd27-75f394567b90" (UID: "325616cd-18f5-448e-bd27-75f394567b90"). InnerVolumeSpecName "kube-api-access-hlp2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:04.679891 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.679784    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util" (OuterVolumeSpecName: "util") pod "325616cd-18f5-448e-bd27-75f394567b90" (UID: "325616cd-18f5-448e-bd27-75f394567b90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:04.680066 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.680043    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util" (OuterVolumeSpecName: "util") pod "0a0cd019-292c-4c35-8069-318eeea30798" (UID: "0a0cd019-292c-4c35-8069-318eeea30798"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:04.772636 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772609    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.772636 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772635    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlp2g\" (UniqueName: \"kubernetes.io/projected/325616cd-18f5-448e-bd27-75f394567b90-kube-api-access-hlp2g\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.772768 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772645    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.772768 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772653    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7tw5\" (UniqueName: \"kubernetes.io/projected/0a0cd019-292c-4c35-8069-318eeea30798-kube-api-access-v7tw5\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.772768 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772661    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/325616cd-18f5-448e-bd27-75f394567b90-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:04.772768 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:04.772668    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a0cd019-292c-4c35-8069-318eeea30798-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:05.458822 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.458789    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn"
Jul 06 19:55:05.459260 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.458821    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn" event={"ID":"0a0cd019-292c-4c35-8069-318eeea30798","Type":"ContainerDied","Data":"62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803"}
Jul 06 19:55:05.459260 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.458870    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62a645365459edaa3f4e55c08206c9426d5cf8df283c1d8fd7d409c8cc4dd803"
Jul 06 19:55:05.460721 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.460694    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk"
Jul 06 19:55:05.460828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.460715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk" event={"ID":"325616cd-18f5-448e-bd27-75f394567b90","Type":"ContainerDied","Data":"49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff"}
Jul 06 19:55:05.460828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.460751    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49e1446bf0a6576f9dc331034cd2a2e42e800c406a553728640752d147d9b8ff"
Jul 06 19:55:05.574387 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.574363    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:05.577148 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.577127    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util\") pod \"831f918f-f685-4667-b76e-1ea3f5af7a91\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") "
Jul 06 19:55:05.577229 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.577176    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle\") pod \"831f918f-f685-4667-b76e-1ea3f5af7a91\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") "
Jul 06 19:55:05.577607 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.577588    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle" (OuterVolumeSpecName: "bundle") pod "831f918f-f685-4667-b76e-1ea3f5af7a91" (UID: "831f918f-f685-4667-b76e-1ea3f5af7a91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:05.581801 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.581770    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util" (OuterVolumeSpecName: "util") pod "831f918f-f685-4667-b76e-1ea3f5af7a91" (UID: "831f918f-f685-4667-b76e-1ea3f5af7a91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:05.677523 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.677482    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rhbv\" (UniqueName: \"kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv\") pod \"831f918f-f685-4667-b76e-1ea3f5af7a91\" (UID: \"831f918f-f685-4667-b76e-1ea3f5af7a91\") "
Jul 06 19:55:05.677711 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.677623    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-util\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:05.677711 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.677665    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/831f918f-f685-4667-b76e-1ea3f5af7a91-bundle\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:05.679502 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.679477    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv" (OuterVolumeSpecName: "kube-api-access-2rhbv") pod "831f918f-f685-4667-b76e-1ea3f5af7a91" (UID: "831f918f-f685-4667-b76e-1ea3f5af7a91"). InnerVolumeSpecName "kube-api-access-2rhbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:05.778205 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:05.778169    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rhbv\" (UniqueName: \"kubernetes.io/projected/831f918f-f685-4667-b76e-1ea3f5af7a91-kube-api-access-2rhbv\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:06.467420 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:06.467386    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm"
Jul 06 19:55:06.467420 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:06.467408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm" event={"ID":"831f918f-f685-4667-b76e-1ea3f5af7a91","Type":"ContainerDied","Data":"9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883"}
Jul 06 19:55:06.467834 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:06.467450    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c0c6ddbb0be8bb7778774435d81eef6ed2498569593bc291a28484a50756883"
Jul 06 19:55:16.029780 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.029747    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"]
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030045    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030056    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030063    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030069    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030078    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030084    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030093    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030098    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="extract"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030105    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030110    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030116    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030121    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030126    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030131    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030136    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030141    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030149    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030154    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030160    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030165    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="pull"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030170    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="util"
Jul 06 19:55:16.030163 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030175    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="util"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030188    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="extract"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030194    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="extract"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030235    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64231890-1a99-4aa1-8c56-5325e0261d05" containerName="extract"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030242    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="831f918f-f685-4667-b76e-1ea3f5af7a91" containerName="extract"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030247    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a0cd019-292c-4c35-8069-318eeea30798" containerName="extract"
Jul 06 19:55:16.030763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.030255    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="325616cd-18f5-448e-bd27-75f394567b90" containerName="extract"
Jul 06 19:55:16.033095 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.033079    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:16.035873 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.035851    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qhlln\""
Jul 06 19:55:16.040998 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.040979    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvjp\" (UniqueName: \"kubernetes.io/projected/484940eb-bdc7-4d96-a8a4-3b449b1d7672-kube-api-access-zfvjp\") pod \"authorino-operator-75cc8d5cb7-slw9h\" (UID: \"484940eb-bdc7-4d96-a8a4-3b449b1d7672\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:16.046730 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.046706    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"]
Jul 06 19:55:16.141944 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.141907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvjp\" (UniqueName: \"kubernetes.io/projected/484940eb-bdc7-4d96-a8a4-3b449b1d7672-kube-api-access-zfvjp\") pod \"authorino-operator-75cc8d5cb7-slw9h\" (UID: \"484940eb-bdc7-4d96-a8a4-3b449b1d7672\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:16.154422 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.154402    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvjp\" (UniqueName: \"kubernetes.io/projected/484940eb-bdc7-4d96-a8a4-3b449b1d7672-kube-api-access-zfvjp\") pod \"authorino-operator-75cc8d5cb7-slw9h\" (UID: \"484940eb-bdc7-4d96-a8a4-3b449b1d7672\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:16.343118 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.343051    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:16.474710 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.474684    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"]
Jul 06 19:55:16.476834 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:16.476807    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484940eb_bdc7_4d96_a8a4_3b449b1d7672.slice/crio-2e476506b700c945f0b79a34eeb2518ad4fe6375d7364f3aaed92107192503e8 WatchSource:0}: Error finding container 2e476506b700c945f0b79a34eeb2518ad4fe6375d7364f3aaed92107192503e8: Status 404 returned error can't find the container with id 2e476506b700c945f0b79a34eeb2518ad4fe6375d7364f3aaed92107192503e8
Jul 06 19:55:16.518452 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:16.518423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h" event={"ID":"484940eb-bdc7-4d96-a8a4-3b449b1d7672","Type":"ContainerStarted","Data":"2e476506b700c945f0b79a34eeb2518ad4fe6375d7364f3aaed92107192503e8"}
Jul 06 19:55:19.215720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.215690    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"]
Jul 06 19:55:19.218846 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.218830    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:19.221351 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.221332    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-k6tvb\""
Jul 06 19:55:19.233229 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.233207    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"]
Jul 06 19:55:19.263005 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.262940    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bvhs\" (UniqueName: \"kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs\") pod \"limitador-operator-controller-manager-859d7b78bd-w7lp7\" (UID: \"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:19.363609 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.363574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bvhs\" (UniqueName: \"kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs\") pod \"limitador-operator-controller-manager-859d7b78bd-w7lp7\" (UID: \"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:19.383521 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.383497    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bvhs\" (UniqueName: \"kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs\") pod \"limitador-operator-controller-manager-859d7b78bd-w7lp7\" (UID: \"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:19.528723 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.528653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:19.536606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.536579    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h" event={"ID":"484940eb-bdc7-4d96-a8a4-3b449b1d7672","Type":"ContainerStarted","Data":"f923631e44cad2a08b84e13ca5fe5279b2b3a2da424ab113a6d7d64dd8416226"}
Jul 06 19:55:19.536715 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.536673    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:19.562013 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.561965    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h" podStartSLOduration=1.030428513 podStartE2EDuration="3.561947462s" totalImagesPullingTime="2.531518949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:19.560549824 +0000 UTC m=+417.822528688" watchObservedRunningTime="2026-07-06 19:55:19.561947462 +0000 UTC m=+417.823926325"
Jul 06 19:55:19.664597 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:19.664575    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"]
Jul 06 19:55:19.666467 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:19.666431    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3f5a48_8b6a_4a91_a7d1_d8ff936fb15b.slice/crio-918852af35d40d08b272aad0ae7cdf7181d5474069f02887de818f589c30dfe9 WatchSource:0}: Error finding container 918852af35d40d08b272aad0ae7cdf7181d5474069f02887de818f589c30dfe9: Status 404 returned error can't find the container with id 918852af35d40d08b272aad0ae7cdf7181d5474069f02887de818f589c30dfe9
Jul 06 19:55:20.543203 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:20.543167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7" event={"ID":"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b","Type":"ContainerStarted","Data":"918852af35d40d08b272aad0ae7cdf7181d5474069f02887de818f589c30dfe9"}
Jul 06 19:55:22.475213 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.475171    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"]
Jul 06 19:55:22.478620 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.478597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:22.481738 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.481718    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jh9nl\""
Jul 06 19:55:22.481999 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.481984    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 19:55:22.491124 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.491099    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpvg\" (UniqueName: \"kubernetes.io/projected/961fa3fe-5cfd-4f70-977d-9c3c6baef362-kube-api-access-7kpvg\") pod \"dns-operator-controller-manager-864dd78c4c-6khmk\" (UID: \"961fa3fe-5cfd-4f70-977d-9c3c6baef362\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:22.494572 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.494549    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"]
Jul 06 19:55:22.559195 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.559168    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7" event={"ID":"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b","Type":"ContainerStarted","Data":"0709782c0f34e96da10cb58a9e8c4f27b6d71b5c692b0ed4d35f976aa025631c"}
Jul 06 19:55:22.559335 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.559303    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:22.581178 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.581132    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7" podStartSLOduration=1.6237916700000001 podStartE2EDuration="3.58111718s" totalImagesPullingTime="1.95732551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:22.579755115 +0000 UTC m=+420.841733980" watchObservedRunningTime="2026-07-06 19:55:22.58111718 +0000 UTC m=+420.843096044"
Jul 06 19:55:22.592212 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.592185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpvg\" (UniqueName: \"kubernetes.io/projected/961fa3fe-5cfd-4f70-977d-9c3c6baef362-kube-api-access-7kpvg\") pod \"dns-operator-controller-manager-864dd78c4c-6khmk\" (UID: \"961fa3fe-5cfd-4f70-977d-9c3c6baef362\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:22.609277 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.609259    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpvg\" (UniqueName: \"kubernetes.io/projected/961fa3fe-5cfd-4f70-977d-9c3c6baef362-kube-api-access-7kpvg\") pod \"dns-operator-controller-manager-864dd78c4c-6khmk\" (UID: \"961fa3fe-5cfd-4f70-977d-9c3c6baef362\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:22.791342 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.791318    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jh9nl\""
Jul 06 19:55:22.798834 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.798807    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:22.933198 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:22.933173    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"]
Jul 06 19:55:22.935092 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:22.935067    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961fa3fe_5cfd_4f70_977d_9c3c6baef362.slice/crio-9764dce86ca377905d6d0702ac94bad799b97c3c8d15d88dafd1f079918dd487 WatchSource:0}: Error finding container 9764dce86ca377905d6d0702ac94bad799b97c3c8d15d88dafd1f079918dd487: Status 404 returned error can't find the container with id 9764dce86ca377905d6d0702ac94bad799b97c3c8d15d88dafd1f079918dd487
Jul 06 19:55:23.565423 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:23.565388    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk" event={"ID":"961fa3fe-5cfd-4f70-977d-9c3c6baef362","Type":"ContainerStarted","Data":"9764dce86ca377905d6d0702ac94bad799b97c3c8d15d88dafd1f079918dd487"}
Jul 06 19:55:25.746346 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:25.746321    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 19:55:26.583669 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:26.583630    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk" event={"ID":"961fa3fe-5cfd-4f70-977d-9c3c6baef362","Type":"ContainerStarted","Data":"af24165e4f7bf307ad86ba612537efe2f2980475df4b3dc4c8fa30bec9969b08"}
Jul 06 19:55:26.583868 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:26.583742    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:26.604292 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:26.604248    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk" podStartSLOduration=1.798216308 podStartE2EDuration="4.604235139s" totalImagesPullingTime="2.806018831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:26.603471182 +0000 UTC m=+424.865450046" watchObservedRunningTime="2026-07-06 19:55:26.604235139 +0000 UTC m=+424.866214059"
Jul 06 19:55:27.391217 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.391185    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"]
Jul 06 19:55:27.394598 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.394580    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.397015 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.397000    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tckhr\""
Jul 06 19:55:27.408750 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.408727    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"]
Jul 06 19:55:27.430250 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.430224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpn58\" (UniqueName: \"kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.430351 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.430260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.530757 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.530732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpn58\" (UniqueName: \"kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.530872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.530766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.531095 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.531080    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.539322 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.539303    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpn58\" (UniqueName: \"kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.705040 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.704962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:27.862031 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:27.862005    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"]
Jul 06 19:55:27.864399 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:27.864377    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd185c00e_ddf4_4c4f_9645_77088ed1b40e.slice/crio-7591118ad3db4c1d251ac7ab575b568513acd8e61c7236f6abef9a55cf3bb53a WatchSource:0}: Error finding container 7591118ad3db4c1d251ac7ab575b568513acd8e61c7236f6abef9a55cf3bb53a: Status 404 returned error can't find the container with id 7591118ad3db4c1d251ac7ab575b568513acd8e61c7236f6abef9a55cf3bb53a
Jul 06 19:55:28.594467 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:28.594434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc" event={"ID":"d185c00e-ddf4-4c4f-9645-77088ed1b40e","Type":"ContainerStarted","Data":"7591118ad3db4c1d251ac7ab575b568513acd8e61c7236f6abef9a55cf3bb53a"}
Jul 06 19:55:30.545176 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:30.545140    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-slw9h"
Jul 06 19:55:33.567651 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:33.567630    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:33.622767 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:33.622733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc" event={"ID":"d185c00e-ddf4-4c4f-9645-77088ed1b40e","Type":"ContainerStarted","Data":"3cd71ccac6db20c00a643a6e3aec1b161b5ba978ff6845a6ca522dfcd83dbfc0"}
Jul 06 19:55:33.622939 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:33.622854    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:33.645877 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:33.645818    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc" podStartSLOduration=1.022269546 podStartE2EDuration="6.645802052s" totalImagesPullingTime="5.623532506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:33.643850982 +0000 UTC m=+431.905829847" watchObservedRunningTime="2026-07-06 19:55:33.645802052 +0000 UTC m=+431.907780918"
Jul 06 19:55:36.391876 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.391845    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"]
Jul 06 19:55:36.396427 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.396409    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.399582 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.399562    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 06 19:55:36.399810 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.399792    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pbbgj\""
Jul 06 19:55:36.399928 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.399882    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 06 19:55:36.418856 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.418832    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"]
Jul 06 19:55:36.503565 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.503533    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.503699 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.503582    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvrt\" (UniqueName: \"kubernetes.io/projected/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-kube-api-access-hwvrt\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.503699 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.503615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.604833 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.604800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvrt\" (UniqueName: \"kubernetes.io/projected/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-kube-api-access-hwvrt\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.604998 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.604850    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.604998 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.604873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.604998 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:36.604985    2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 06 19:55:36.605125 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:36.605057    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert podName:ff4a89fd-6a0f-4abb-8721-fb90346e7e76 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:37.105039646 +0000 UTC m=+435.367018494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-bn2fn" (UID: "ff4a89fd-6a0f-4abb-8721-fb90346e7e76") : secret "plugin-serving-cert" not found
Jul 06 19:55:36.605428 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.605412    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:36.613812 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:36.613789    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvrt\" (UniqueName: \"kubernetes.io/projected/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-kube-api-access-hwvrt\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:37.107889 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.107845    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:37.110290 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.110259    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff4a89fd-6a0f-4abb-8721-fb90346e7e76-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bn2fn\" (UID: \"ff4a89fd-6a0f-4abb-8721-fb90346e7e76\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:37.241321 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.241289    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-558b86594d-m6h5m"]
Jul 06 19:55:37.244882 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.244863    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.249487 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249453    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9f9bq\""
Jul 06 19:55:37.249487 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249470    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 19:55:37.249656 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249461    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 19:55:37.249656 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249461    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 19:55:37.249656 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249540    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 19:55:37.249867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249748    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 19:55:37.249867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249822    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 19:55:37.249867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.249842    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 19:55:37.253804 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.253784    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 19:55:37.256346 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.256327    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-558b86594d-m6h5m"]
Jul 06 19:55:37.306805 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.306780    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"
Jul 06 19:55:37.309477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-oauth-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309562 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309492    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-service-ca\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309562 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309518    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvf4\" (UniqueName: \"kubernetes.io/projected/9e752371-bad5-4189-b549-af098ff8c2aa-kube-api-access-8xvf4\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309970 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-oauth-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309970 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309970 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-console-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.309970 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.309751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-trusted-ca-bundle\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-console-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-trusted-ca-bundle\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-oauth-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410553    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-service-ca\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvf4\" (UniqueName: \"kubernetes.io/projected/9e752371-bad5-4189-b549-af098ff8c2aa-kube-api-access-8xvf4\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-oauth-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.410872 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.410786    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.411300 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.411272    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-console-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.411397 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.411340    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-oauth-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.411397 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.411371    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-trusted-ca-bundle\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.411772 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.411751    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752371-bad5-4189-b549-af098ff8c2aa-service-ca\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.413565 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.413537    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-oauth-config\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.413993 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.413973    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e752371-bad5-4189-b549-af098ff8c2aa-console-serving-cert\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.418742 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.418721    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvf4\" (UniqueName: \"kubernetes.io/projected/9e752371-bad5-4189-b549-af098ff8c2aa-kube-api-access-8xvf4\") pod \"console-558b86594d-m6h5m\" (UID: \"9e752371-bad5-4189-b549-af098ff8c2aa\") " pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.434073 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.434054    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn"]
Jul 06 19:55:37.435969 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:37.435949    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff4a89fd_6a0f_4abb_8721_fb90346e7e76.slice/crio-5714ba4f4e7c5c63c3d496170465666c4cc6c463a6039f7171e7857bda731fa3 WatchSource:0}: Error finding container 5714ba4f4e7c5c63c3d496170465666c4cc6c463a6039f7171e7857bda731fa3: Status 404 returned error can't find the container with id 5714ba4f4e7c5c63c3d496170465666c4cc6c463a6039f7171e7857bda731fa3
Jul 06 19:55:37.555113 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.555093    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:37.590961 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.590935    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6khmk"
Jul 06 19:55:37.647464 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.647429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn" event={"ID":"ff4a89fd-6a0f-4abb-8721-fb90346e7e76","Type":"ContainerStarted","Data":"5714ba4f4e7c5c63c3d496170465666c4cc6c463a6039f7171e7857bda731fa3"}
Jul 06 19:55:37.691479 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:37.691458    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-558b86594d-m6h5m"]
Jul 06 19:55:37.693120 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:55:37.693094    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e752371_bad5_4189_b549_af098ff8c2aa.slice/crio-e20616f794515b2f37355077b79dddabd3134ddadeb1d3d96473a804e186de07 WatchSource:0}: Error finding container e20616f794515b2f37355077b79dddabd3134ddadeb1d3d96473a804e186de07: Status 404 returned error can't find the container with id e20616f794515b2f37355077b79dddabd3134ddadeb1d3d96473a804e186de07
Jul 06 19:55:38.655791 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:38.655757    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558b86594d-m6h5m" event={"ID":"9e752371-bad5-4189-b549-af098ff8c2aa","Type":"ContainerStarted","Data":"693981a26e379211cbd831a0bfcf92855775fde4f49f08ba69c37c57d6a5c7e0"}
Jul 06 19:55:38.656264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:38.655812    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558b86594d-m6h5m" event={"ID":"9e752371-bad5-4189-b549-af098ff8c2aa","Type":"ContainerStarted","Data":"e20616f794515b2f37355077b79dddabd3134ddadeb1d3d96473a804e186de07"}
Jul 06 19:55:38.676639 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:38.676585    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-558b86594d-m6h5m" podStartSLOduration=1.676566309 podStartE2EDuration="1.676566309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:38.674664749 +0000 UTC m=+436.936643614" watchObservedRunningTime="2026-07-06 19:55:38.676566309 +0000 UTC m=+436.938545166"
Jul 06 19:55:44.631448 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:44.631363    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.451828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.451791    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"]
Jul 06 19:55:46.452304 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.452035    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" containerName="manager" containerID="cri-o://3cd71ccac6db20c00a643a6e3aec1b161b5ba978ff6845a6ca522dfcd83dbfc0" gracePeriod=2
Jul 06 19:55:46.455056 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.455021    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.458400 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.458364    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"]
Jul 06 19:55:46.469183 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.469156    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"]
Jul 06 19:55:46.469371 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.469345    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" containerName="manager" containerID="cri-o://0709782c0f34e96da10cb58a9e8c4f27b6d71b5c692b0ed4d35f976aa025631c" gracePeriod=2
Jul 06 19:55:46.475558 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.475537    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:55:46.477031 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.476751    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" containerName="manager"
Jul 06 19:55:46.477031 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.476778    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" containerName="manager"
Jul 06 19:55:46.481453 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.479517    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" containerName="manager"
Jul 06 19:55:46.481740 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.481695    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.484047 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.484021    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"]
Jul 06 19:55:46.484165 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.484140    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.490826 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.490798    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w7lp7\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:46.493619 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.493597    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:55:46.494153 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.494121    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.501193 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.501174    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"]
Jul 06 19:55:46.501451 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.501434    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" containerName="manager"
Jul 06 19:55:46.501451 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.501450    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" containerName="manager"
Jul 06 19:55:46.501594 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.501548    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" containerName="manager"
Jul 06 19:55:46.504490 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.504472    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:55:46.531822 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.531800    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"]
Jul 06 19:55:46.576312 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.576276    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w7lp7\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:46.578835 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:55:46.578807    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prpdc\" is forbidden: User \"system:node:ip-10-0-132-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-142.ec2.internal' and this object" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.598606 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.598540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.598728 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.598607    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4swsx\" (UniqueName: \"kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.598728 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.598656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k8d7\" (UniqueName: \"kubernetes.io/projected/15a41293-96b5-4785-bf37-df3800d321f2-kube-api-access-8k8d7\") pod \"limitador-operator-controller-manager-859d7b78bd-jb9j5\" (UID: \"15a41293-96b5-4785-bf37-df3800d321f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:55:46.699342 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.699301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.699512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.699380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4swsx\" (UniqueName: \"kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.699512 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.699433    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k8d7\" (UniqueName: \"kubernetes.io/projected/15a41293-96b5-4785-bf37-df3800d321f2-kube-api-access-8k8d7\") pod \"limitador-operator-controller-manager-859d7b78bd-jb9j5\" (UID: \"15a41293-96b5-4785-bf37-df3800d321f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:55:46.699994 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.699971    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.701034 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.700982    2565 generic.go:356] "Generic (PLEG): container finished" podID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" containerID="3cd71ccac6db20c00a643a6e3aec1b161b5ba978ff6845a6ca522dfcd83dbfc0" exitCode=0
Jul 06 19:55:46.703330 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.703271    2565 generic.go:356] "Generic (PLEG): container finished" podID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" containerID="0709782c0f34e96da10cb58a9e8c4f27b6d71b5c692b0ed4d35f976aa025631c" exitCode=0
Jul 06 19:55:46.714838 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.714789    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4swsx\" (UniqueName: \"kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m7kg2\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.715760 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.715734    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k8d7\" (UniqueName: \"kubernetes.io/projected/15a41293-96b5-4785-bf37-df3800d321f2-kube-api-access-8k8d7\") pod \"limitador-operator-controller-manager-859d7b78bd-jb9j5\" (UID: \"15a41293-96b5-4785-bf37-df3800d321f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:55:46.737546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.737523    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:46.740312 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.740294    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:46.799875 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.799841    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bvhs\" (UniqueName: \"kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs\") pod \"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b\" (UID: \"5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b\") "
Jul 06 19:55:46.800068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.799891    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume\") pod \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") "
Jul 06 19:55:46.800068 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.799969    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpn58\" (UniqueName: \"kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58\") pod \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\" (UID: \"d185c00e-ddf4-4c4f-9645-77088ed1b40e\") "
Jul 06 19:55:46.800546 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.800498    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d185c00e-ddf4-4c4f-9645-77088ed1b40e" (UID: "d185c00e-ddf4-4c4f-9645-77088ed1b40e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:55:46.802615 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.802586    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs" (OuterVolumeSpecName: "kube-api-access-6bvhs") pod "5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" (UID: "5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b"). InnerVolumeSpecName "kube-api-access-6bvhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:46.802732 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.802586    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58" (OuterVolumeSpecName: "kube-api-access-zpn58") pod "d185c00e-ddf4-4c4f-9645-77088ed1b40e" (UID: "d185c00e-ddf4-4c4f-9645-77088ed1b40e"). InnerVolumeSpecName "kube-api-access-zpn58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:55:46.889208 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.889171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:55:46.897164 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.897128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:55:46.901267 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.901241    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpn58\" (UniqueName: \"kubernetes.io/projected/d185c00e-ddf4-4c4f-9645-77088ed1b40e-kube-api-access-zpn58\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:46.901358 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.901274    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bvhs\" (UniqueName: \"kubernetes.io/projected/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b-kube-api-access-6bvhs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:46.901358 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:46.901290    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d185c00e-ddf4-4c4f-9645-77088ed1b40e-extensions-socket-volume\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:55:47.555511 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.555475    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:47.555963 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.555540    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:47.561445 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.561421    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:47.710986 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.710956    2565 scope.go:117] "RemoveContainer" containerID="0709782c0f34e96da10cb58a9e8c4f27b6d71b5c692b0ed4d35f976aa025631c"
Jul 06 19:55:47.710986 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.710972    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w7lp7"
Jul 06 19:55:47.712737 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.712715    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prpdc"
Jul 06 19:55:47.717788 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:47.717720    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-558b86594d-m6h5m"
Jul 06 19:55:48.655519 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:48.655483    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b" path="/var/lib/kubelet/pods/5b3f5a48-8b6a-4a91-a7d1-d8ff936fb15b/volumes"
Jul 06 19:55:48.655929 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:55:48.655879    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d185c00e-ddf4-4c4f-9645-77088ed1b40e" path="/var/lib/kubelet/pods/d185c00e-ddf4-4c4f-9645-77088ed1b40e/volumes"
Jul 06 19:56:00.623417 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:00.623107    2565 scope.go:117] "RemoveContainer" containerID="3cd71ccac6db20c00a643a6e3aec1b161b5ba978ff6845a6ca522dfcd83dbfc0"
Jul 06 19:56:00.774902 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:00.774875    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:56:00.775453 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:56:00.775427    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73b3824_677e_4e76_994a_f1fa7796feec.slice/crio-86433ad99ab2f1ce4c4d2853f429a03dfd11f49aa2754340933ef446f396d980 WatchSource:0}: Error finding container 86433ad99ab2f1ce4c4d2853f429a03dfd11f49aa2754340933ef446f396d980: Status 404 returned error can't find the container with id 86433ad99ab2f1ce4c4d2853f429a03dfd11f49aa2754340933ef446f396d980
Jul 06 19:56:00.791125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:00.790961    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"]
Jul 06 19:56:00.791338 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:00.791313    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" event={"ID":"f73b3824-677e-4e76-994a-f1fa7796feec","Type":"ContainerStarted","Data":"86433ad99ab2f1ce4c4d2853f429a03dfd11f49aa2754340933ef446f396d980"}
Jul 06 19:56:00.794281 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:56:00.794255    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a41293_96b5_4785_bf37_df3800d321f2.slice/crio-2fb0bcc5fb43a289685e6b080905eb2186b8cdae9f120adeb3b17c3c2c62327e WatchSource:0}: Error finding container 2fb0bcc5fb43a289685e6b080905eb2186b8cdae9f120adeb3b17c3c2c62327e: Status 404 returned error can't find the container with id 2fb0bcc5fb43a289685e6b080905eb2186b8cdae9f120adeb3b17c3c2c62327e
Jul 06 19:56:01.806048 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.806008    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn" event={"ID":"ff4a89fd-6a0f-4abb-8721-fb90346e7e76","Type":"ContainerStarted","Data":"5aa23dd357ae255976b220242d82bac87556be24f8162b7f97365aebea0a0366"}
Jul 06 19:56:01.807646 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.807621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" event={"ID":"f73b3824-677e-4e76-994a-f1fa7796feec","Type":"ContainerStarted","Data":"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"}
Jul 06 19:56:01.807759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.807693    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:56:01.809108 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.809086    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5" event={"ID":"15a41293-96b5-4785-bf37-df3800d321f2","Type":"ContainerStarted","Data":"c800bdfe60bf654052f9f0967858a28d8d53595e69484acb3f3dbeb9ef781157"}
Jul 06 19:56:01.809200 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.809123    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5" event={"ID":"15a41293-96b5-4785-bf37-df3800d321f2","Type":"ContainerStarted","Data":"2fb0bcc5fb43a289685e6b080905eb2186b8cdae9f120adeb3b17c3c2c62327e"}
Jul 06 19:56:01.809200 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.809176    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:56:01.823407 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.823365    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bn2fn" podStartSLOduration=2.537616269 podStartE2EDuration="25.823350469s" totalImagesPullingTime="23.2857342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:01.822430375 +0000 UTC m=+460.084409239" watchObservedRunningTime="2026-07-06 19:56:01.823350469 +0000 UTC m=+460.085329333"
Jul 06 19:56:01.842094 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.842034    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5" podStartSLOduration=15.842012651 podStartE2EDuration="15.842012651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:01.839337993 +0000 UTC m=+460.101316859" watchObservedRunningTime="2026-07-06 19:56:01.842012651 +0000 UTC m=+460.103991517"
Jul 06 19:56:01.858861 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:01.858811    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" podStartSLOduration=15.858795157 podStartE2EDuration="15.858795157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:01.857291663 +0000 UTC m=+460.119270563" watchObservedRunningTime="2026-07-06 19:56:01.858795157 +0000 UTC m=+460.120774020"
Jul 06 19:56:03.195341 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.195298    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:56:03.446999 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.446897    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 19:56:03.450317 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.450293    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.456125 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.456100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.456264 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.456138    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwmrs\" (UniqueName: \"kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.466935 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.466901    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 19:56:03.557111 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.557066    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.557308 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.557122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwmrs\" (UniqueName: \"kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.557509 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.557488    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.566277 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.566244    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwmrs\" (UniqueName: \"kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs\") pod \"kuadrant-operator-controller-manager-86878c769f-fjx7s\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.760301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.760269    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:03.821367 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.821302    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" podUID="f73b3824-677e-4e76-994a-f1fa7796feec" containerName="manager" containerID="cri-o://3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9" gracePeriod=10
Jul 06 19:56:03.910342 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:03.910306    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 19:56:03.913957 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:56:03.913930    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53940790_9a66_455b_886c_02decc68c7c8.slice/crio-16e8d380bdaca8c2ff7dfc0cafa147a2dbe704b68ca920ab209b38a4c694974d WatchSource:0}: Error finding container 16e8d380bdaca8c2ff7dfc0cafa147a2dbe704b68ca920ab209b38a4c694974d: Status 404 returned error can't find the container with id 16e8d380bdaca8c2ff7dfc0cafa147a2dbe704b68ca920ab209b38a4c694974d
Jul 06 19:56:04.056585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.056559    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:56:04.062237 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.062217    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4swsx\" (UniqueName: \"kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx\") pod \"f73b3824-677e-4e76-994a-f1fa7796feec\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") "
Jul 06 19:56:04.062323 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.062283    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume\") pod \"f73b3824-677e-4e76-994a-f1fa7796feec\" (UID: \"f73b3824-677e-4e76-994a-f1fa7796feec\") "
Jul 06 19:56:04.062568 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.062548    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f73b3824-677e-4e76-994a-f1fa7796feec" (UID: "f73b3824-677e-4e76-994a-f1fa7796feec"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:56:04.064138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.064115    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx" (OuterVolumeSpecName: "kube-api-access-4swsx") pod "f73b3824-677e-4e76-994a-f1fa7796feec" (UID: "f73b3824-677e-4e76-994a-f1fa7796feec"). InnerVolumeSpecName "kube-api-access-4swsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:56:04.163673 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.163641    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73b3824-677e-4e76-994a-f1fa7796feec-extensions-socket-volume\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:56:04.163673 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.163671    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4swsx\" (UniqueName: \"kubernetes.io/projected/f73b3824-677e-4e76-994a-f1fa7796feec-kube-api-access-4swsx\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:56:04.829462 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.829361    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" event={"ID":"53940790-9a66-455b-886c-02decc68c7c8","Type":"ContainerStarted","Data":"a88ac29e9c0b07cc50d3de5c01023990d991b4520ea0fbce7cdd3980791af4cf"}
Jul 06 19:56:04.829462 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.829423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" event={"ID":"53940790-9a66-455b-886c-02decc68c7c8","Type":"ContainerStarted","Data":"16e8d380bdaca8c2ff7dfc0cafa147a2dbe704b68ca920ab209b38a4c694974d"}
Jul 06 19:56:04.829959 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.829521    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:04.830878 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.830848    2565 generic.go:356] "Generic (PLEG): container finished" podID="f73b3824-677e-4e76-994a-f1fa7796feec" containerID="3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9" exitCode=0
Jul 06 19:56:04.831010 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.830888    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"
Jul 06 19:56:04.831010 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.830887    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" event={"ID":"f73b3824-677e-4e76-994a-f1fa7796feec","Type":"ContainerDied","Data":"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"}
Jul 06 19:56:04.831010 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.830950    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2" event={"ID":"f73b3824-677e-4e76-994a-f1fa7796feec","Type":"ContainerDied","Data":"86433ad99ab2f1ce4c4d2853f429a03dfd11f49aa2754340933ef446f396d980"}
Jul 06 19:56:04.831010 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.830972    2565 scope.go:117] "RemoveContainer" containerID="3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"
Jul 06 19:56:04.839671 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.839653    2565 scope.go:117] "RemoveContainer" containerID="3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"
Jul 06 19:56:04.839947 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:56:04.839928    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9\": container with ID starting with 3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9 not found: ID does not exist" containerID="3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"
Jul 06 19:56:04.840011 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.839960    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9"} err="failed to get container status \"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9\": rpc error: code = NotFound desc = could not find container \"3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9\": container with ID starting with 3aaf30a640f179c2c16d3fb8b122027ec14707b20a6c10f0e0f8471320345af9 not found: ID does not exist"
Jul 06 19:56:04.852969 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.852925    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" podStartSLOduration=1.852894526 podStartE2EDuration="1.852894526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:04.851129435 +0000 UTC m=+463.113108298" watchObservedRunningTime="2026-07-06 19:56:04.852894526 +0000 UTC m=+463.114873391"
Jul 06 19:56:04.871644 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.871593    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:56:04.873843 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:04.873825    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m7kg2"]
Jul 06 19:56:06.655191 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:06.655160    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73b3824-677e-4e76-994a-f1fa7796feec" path="/var/lib/kubelet/pods/f73b3824-677e-4e76-994a-f1fa7796feec/volumes"
Jul 06 19:56:12.816615 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:12.816586    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jb9j5"
Jul 06 19:56:15.839287 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:15.839254    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 19:56:33.293108 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.293076    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:56:33.293554 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.293438    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73b3824-677e-4e76-994a-f1fa7796feec" containerName="manager"
Jul 06 19:56:33.293554 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.293450    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73b3824-677e-4e76-994a-f1fa7796feec" containerName="manager"
Jul 06 19:56:33.293554 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.293509    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73b3824-677e-4e76-994a-f1fa7796feec" containerName="manager"
Jul 06 19:56:33.344228 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.344190    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:56:33.344385 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.344243    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:56:33.344385 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.344323    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.346660 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.346639    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 19:56:33.392035 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.392006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-config-file\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.392154 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.392091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm6lj\" (UniqueName: \"kubernetes.io/projected/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-kube-api-access-pm6lj\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.492730 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.492705    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm6lj\" (UniqueName: \"kubernetes.io/projected/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-kube-api-access-pm6lj\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.492867 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.492754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-config-file\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.493418 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.493401    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-config-file\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.501637 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.501617    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm6lj\" (UniqueName: \"kubernetes.io/projected/5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef-kube-api-access-pm6lj\") pod \"limitador-limitador-54d6d988d5-mts8f\" (UID: \"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.655054 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.654984    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:33.809426 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.809401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:56:33.811961 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:56:33.811928    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f61059a_00b0_4cb9_a2b3_67eb1f6b35ef.slice/crio-cc88c01101d2107d96f0c0d60270e3da6c44d905c348ee787b8e37a854006ba1 WatchSource:0}: Error finding container cc88c01101d2107d96f0c0d60270e3da6c44d905c348ee787b8e37a854006ba1: Status 404 returned error can't find the container with id cc88c01101d2107d96f0c0d60270e3da6c44d905c348ee787b8e37a854006ba1
Jul 06 19:56:33.878301 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.878160    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:33.890869 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.890843    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:33.890988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.890922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:33.893464 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.893446    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-svfj7\""
Jul 06 19:56:33.976329 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.976268    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f" event={"ID":"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef","Type":"ContainerStarted","Data":"cc88c01101d2107d96f0c0d60270e3da6c44d905c348ee787b8e37a854006ba1"}
Jul 06 19:56:33.997218 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:33.997190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqcz\" (UniqueName: \"kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz\") pod \"authorino-58cd955675-rbrnr\" (UID: \"d14c266e-8e71-404f-b159-3767cd76cf04\") " pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:34.097590 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:34.097558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqcz\" (UniqueName: \"kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz\") pod \"authorino-58cd955675-rbrnr\" (UID: \"d14c266e-8e71-404f-b159-3767cd76cf04\") " pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:34.106477 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:34.106453    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqcz\" (UniqueName: \"kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz\") pod \"authorino-58cd955675-rbrnr\" (UID: \"d14c266e-8e71-404f-b159-3767cd76cf04\") " pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:34.201004 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:34.200975    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:34.457757 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:34.457726    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:34.461059 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:56:34.461032    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd14c266e_8e71_404f_b159_3767cd76cf04.slice/crio-8f81779ca948bca20ddbe25769cbb3d7cd5914c036cc714be8e2ee26ecd4162d WatchSource:0}: Error finding container 8f81779ca948bca20ddbe25769cbb3d7cd5914c036cc714be8e2ee26ecd4162d: Status 404 returned error can't find the container with id 8f81779ca948bca20ddbe25769cbb3d7cd5914c036cc714be8e2ee26ecd4162d
Jul 06 19:56:34.982674 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:34.982635    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbrnr" event={"ID":"d14c266e-8e71-404f-b159-3767cd76cf04","Type":"ContainerStarted","Data":"8f81779ca948bca20ddbe25769cbb3d7cd5914c036cc714be8e2ee26ecd4162d"}
Jul 06 19:56:38.396739 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:38.396651    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:39.007991 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:39.007941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f" event={"ID":"5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef","Type":"ContainerStarted","Data":"bb1668e1e332cdf7813b7086eb6a7f6eb4bf3a5585298cd1478840d7876592d3"}
Jul 06 19:56:39.008184 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:39.008107    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:56:39.009345 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:39.009319    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbrnr" event={"ID":"d14c266e-8e71-404f-b159-3767cd76cf04","Type":"ContainerStarted","Data":"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"}
Jul 06 19:56:39.027430 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:39.027391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f" podStartSLOduration=1.725928109 podStartE2EDuration="6.027377668s" totalImagesPullingTime="4.301449559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:39.025793321 +0000 UTC m=+497.287772186" watchObservedRunningTime="2026-07-06 19:56:39.027377668 +0000 UTC m=+497.289356531"
Jul 06 19:56:39.047191 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:39.047152    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rbrnr" podStartSLOduration=2.334349579 podStartE2EDuration="6.04713957s" totalImagesPullingTime="3.712789991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:39.045525757 +0000 UTC m=+497.307504624" watchObservedRunningTime="2026-07-06 19:56:39.04713957 +0000 UTC m=+497.309118434"
Jul 06 19:56:40.015426 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:40.015382    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rbrnr" podUID="d14c266e-8e71-404f-b159-3767cd76cf04" containerName="authorino" containerID="cri-o://5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568" gracePeriod=30
Jul 06 19:56:40.267437 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:40.267379    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:40.352162 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:40.352136    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqqcz\" (UniqueName: \"kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz\") pod \"d14c266e-8e71-404f-b159-3767cd76cf04\" (UID: \"d14c266e-8e71-404f-b159-3767cd76cf04\") "
Jul 06 19:56:40.354158 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:40.354134    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz" (OuterVolumeSpecName: "kube-api-access-wqqcz") pod "d14c266e-8e71-404f-b159-3767cd76cf04" (UID: "d14c266e-8e71-404f-b159-3767cd76cf04"). InnerVolumeSpecName "kube-api-access-wqqcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:56:40.452846 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:40.452820    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqqcz\" (UniqueName: \"kubernetes.io/projected/d14c266e-8e71-404f-b159-3767cd76cf04-kube-api-access-wqqcz\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:56:41.021439 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.021408    2565 generic.go:356] "Generic (PLEG): container finished" podID="d14c266e-8e71-404f-b159-3767cd76cf04" containerID="5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568" exitCode=0
Jul 06 19:56:41.021863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.021447    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbrnr"
Jul 06 19:56:41.021863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.021459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbrnr" event={"ID":"d14c266e-8e71-404f-b159-3767cd76cf04","Type":"ContainerDied","Data":"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"}
Jul 06 19:56:41.021863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.021490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbrnr" event={"ID":"d14c266e-8e71-404f-b159-3767cd76cf04","Type":"ContainerDied","Data":"8f81779ca948bca20ddbe25769cbb3d7cd5914c036cc714be8e2ee26ecd4162d"}
Jul 06 19:56:41.021863 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.021506    2565 scope.go:117] "RemoveContainer" containerID="5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"
Jul 06 19:56:41.030080 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.030063    2565 scope.go:117] "RemoveContainer" containerID="5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"
Jul 06 19:56:41.030347 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:56:41.030312    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568\": container with ID starting with 5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568 not found: ID does not exist" containerID="5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"
Jul 06 19:56:41.030415 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.030358    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568"} err="failed to get container status \"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568\": rpc error: code = NotFound desc = could not find container \"5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568\": container with ID starting with 5d7a5ef781b7f9c4103c20efc3deb263e57e85b2090d9cd6c47db222e09ea568 not found: ID does not exist"
Jul 06 19:56:41.038593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.038572    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:41.047116 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:41.043991    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbrnr"]
Jul 06 19:56:42.655025 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:42.654995    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d14c266e-8e71-404f-b159-3767cd76cf04" path="/var/lib/kubelet/pods/d14c266e-8e71-404f-b159-3767cd76cf04/volumes"
Jul 06 19:56:50.015987 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:56:50.015956    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mts8f"
Jul 06 19:57:05.026605 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.026566    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rbhf2"]
Jul 06 19:57:05.027240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.027096    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14c266e-8e71-404f-b159-3767cd76cf04" containerName="authorino"
Jul 06 19:57:05.027240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.027117    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14c266e-8e71-404f-b159-3767cd76cf04" containerName="authorino"
Jul 06 19:57:05.027240 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.027217    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d14c266e-8e71-404f-b159-3767cd76cf04" containerName="authorino"
Jul 06 19:57:05.030374 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.030354    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:05.032712 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.032689    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-svfj7\""
Jul 06 19:57:05.038059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.038036    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rbhf2"]
Jul 06 19:57:05.155139 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.155105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8mmc\" (UniqueName: \"kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc\") pod \"authorino-5d4b67c446-rbhf2\" (UID: \"18871b0d-6650-4c60-b296-2d461fd9adfe\") " pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:05.256114 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.256083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8mmc\" (UniqueName: \"kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc\") pod \"authorino-5d4b67c446-rbhf2\" (UID: \"18871b0d-6650-4c60-b296-2d461fd9adfe\") " pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:05.261922 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.261878    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rbhf2"]
Jul 06 19:57:05.262096 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:57:05.262078    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v8mmc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-rbhf2" podUID="18871b0d-6650-4c60-b296-2d461fd9adfe"
Jul 06 19:57:05.264343 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.264322    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8mmc\" (UniqueName: \"kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc\") pod \"authorino-5d4b67c446-rbhf2\" (UID: \"18871b0d-6650-4c60-b296-2d461fd9adfe\") " pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:05.283598 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.283534    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c8c884ddc-flkkp"]
Jul 06 19:57:05.289289 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.289271    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:05.296720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.296693    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c8c884ddc-flkkp"]
Jul 06 19:57:05.357467 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.357436    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rn5\" (UniqueName: \"kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5\") pod \"authorino-7c8c884ddc-flkkp\" (UID: \"830d912f-9967-4f27-afea-7f8402ee9ef1\") " pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:05.387994 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.387966    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c8c884ddc-flkkp"]
Jul 06 19:57:05.388281 ip-10-0-132-142 kubenswrapper[2565]: E0706 19:57:05.388233    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-75rn5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7c8c884ddc-flkkp" podUID="830d912f-9967-4f27-afea-7f8402ee9ef1"
Jul 06 19:57:05.458702 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.458669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rn5\" (UniqueName: \"kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5\") pod \"authorino-7c8c884ddc-flkkp\" (UID: \"830d912f-9967-4f27-afea-7f8402ee9ef1\") " pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:05.466220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:05.466200    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rn5\" (UniqueName: \"kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5\") pod \"authorino-7c8c884ddc-flkkp\" (UID: \"830d912f-9967-4f27-afea-7f8402ee9ef1\") " pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:06.144692 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.144662    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:06.145085 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.144662    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:06.152352 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.152330    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:06.155174 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.155158    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:06.163966 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.163947    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75rn5\" (UniqueName: \"kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5\") pod \"830d912f-9967-4f27-afea-7f8402ee9ef1\" (UID: \"830d912f-9967-4f27-afea-7f8402ee9ef1\") "
Jul 06 19:57:06.164075 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.163986    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8mmc\" (UniqueName: \"kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc\") pod \"18871b0d-6650-4c60-b296-2d461fd9adfe\" (UID: \"18871b0d-6650-4c60-b296-2d461fd9adfe\") "
Jul 06 19:57:06.165983 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.165957    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5" (OuterVolumeSpecName: "kube-api-access-75rn5") pod "830d912f-9967-4f27-afea-7f8402ee9ef1" (UID: "830d912f-9967-4f27-afea-7f8402ee9ef1"). InnerVolumeSpecName "kube-api-access-75rn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:57:06.166084 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.165992    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc" (OuterVolumeSpecName: "kube-api-access-v8mmc") pod "18871b0d-6650-4c60-b296-2d461fd9adfe" (UID: "18871b0d-6650-4c60-b296-2d461fd9adfe"). InnerVolumeSpecName "kube-api-access-v8mmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:57:06.264503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.264478    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75rn5\" (UniqueName: \"kubernetes.io/projected/830d912f-9967-4f27-afea-7f8402ee9ef1-kube-api-access-75rn5\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:57:06.264503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:06.264500    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8mmc\" (UniqueName: \"kubernetes.io/projected/18871b0d-6650-4c60-b296-2d461fd9adfe-kube-api-access-v8mmc\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 19:57:07.149483 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.149447    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rbhf2"
Jul 06 19:57:07.149885 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.149447    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c8c884ddc-flkkp"
Jul 06 19:57:07.185447 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.185411    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rbhf2"]
Jul 06 19:57:07.199425 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.199394    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rbhf2"]
Jul 06 19:57:07.240551 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.240517    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c8c884ddc-flkkp"]
Jul 06 19:57:07.251317 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:07.251290    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c8c884ddc-flkkp"]
Jul 06 19:57:08.655223 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:08.655194    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18871b0d-6650-4c60-b296-2d461fd9adfe" path="/var/lib/kubelet/pods/18871b0d-6650-4c60-b296-2d461fd9adfe/volumes"
Jul 06 19:57:08.655590 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:08.655399    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830d912f-9967-4f27-afea-7f8402ee9ef1" path="/var/lib/kubelet/pods/830d912f-9967-4f27-afea-7f8402ee9ef1/volumes"
Jul 06 19:57:48.972078 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:57:48.972041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:58:13.063614 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:13.063579    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:58:22.578938 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:22.578892    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:58:22.582379 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:22.582358    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 19:58:23.198066 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.198039    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"]
Jul 06 19:58:23.201138 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.201120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.203952 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.203927    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 19:58:23.204095 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.204078    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6wr7c\""
Jul 06 19:58:23.205140 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.205123    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 19:58:23.205242 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.205221    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 06 19:58:23.215250 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.215226    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"]
Jul 06 19:58:23.255818 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.255782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqw6\" (UniqueName: \"kubernetes.io/projected/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-kube-api-access-zmqw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.255988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.255827    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.255988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.255850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.255988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.255891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.255988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.255970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.256182 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.256045    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356448 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356418    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqw6\" (UniqueName: \"kubernetes.io/projected/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-kube-api-access-zmqw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356557    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.356852 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.357032 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.357014    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.357105 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356990    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.357105 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.356962    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.358887 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.358864    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.359462 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.359434    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.368527 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.368505    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqw6\" (UniqueName: \"kubernetes.io/projected/085ee9ab-bac0-4646-86cb-3eba8fc3e5c8-kube-api-access-zmqw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fsbff\" (UID: \"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.511894 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.511856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:23.667777 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:58:23.667738    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085ee9ab_bac0_4646_86cb_3eba8fc3e5c8.slice/crio-4f01a55e7f737df0a5dbcd5908b2debe038fd0961047be259e45756f8d7aa725 WatchSource:0}: Error finding container 4f01a55e7f737df0a5dbcd5908b2debe038fd0961047be259e45756f8d7aa725: Status 404 returned error can't find the container with id 4f01a55e7f737df0a5dbcd5908b2debe038fd0961047be259e45756f8d7aa725
Jul 06 19:58:23.668777 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.668751    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"]
Jul 06 19:58:23.985024 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:23.984931    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:58:24.554491 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:24.554445    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff" event={"ID":"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8","Type":"ContainerStarted","Data":"4f01a55e7f737df0a5dbcd5908b2debe038fd0961047be259e45756f8d7aa725"}
Jul 06 19:58:25.561199 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:25.561152    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff" event={"ID":"085ee9ab-bac0-4646-86cb-3eba8fc3e5c8","Type":"ContainerStarted","Data":"f3daf216722720d8b1fccb0b1c45633a148135fadc443cadc9b1dc140de56d75"}
Jul 06 19:58:25.580868 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:25.580819    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff" podStartSLOduration=1.545680189 podStartE2EDuration="2.580804293s" totalImagesPullingTime="1.035124104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:58:25.578875822 +0000 UTC m=+603.840854685" watchObservedRunningTime="2026-07-06 19:58:25.580804293 +0000 UTC m=+603.842783156"
Jul 06 19:58:33.512459 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:33.512420    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:33.512834 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:33.512512    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:33.524970 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:33.524949    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:33.611680 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:33.611650    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fsbff"
Jul 06 19:58:35.404195 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.404164    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"]
Jul 06 19:58:35.408109 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.408083    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.410632 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.410607    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 06 19:58:35.419236 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.419214    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"]
Jul 06 19:58:35.446463 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.446593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446467    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f77jj\" (UniqueName: \"kubernetes.io/projected/ffb21700-4982-471f-82c5-f5e7a7803d3e-kube-api-access-f77jj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.446593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446490    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.446593 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.446697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.446697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.446625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb21700-4982-471f-82c5-f5e7a7803d3e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547054 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547063    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb21700-4982-471f-82c5-f5e7a7803d3e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f77jj\" (UniqueName: \"kubernetes.io/projected/ffb21700-4982-471f-82c5-f5e7a7803d3e-kube-api-access-f77jj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547220 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547503 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547482    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547543 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547521    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.547585 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.547565    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.549441 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.549418    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb21700-4982-471f-82c5-f5e7a7803d3e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.549614 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.549595    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb21700-4982-471f-82c5-f5e7a7803d3e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.556334 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.556314    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f77jj\" (UniqueName: \"kubernetes.io/projected/ffb21700-4982-471f-82c5-f5e7a7803d3e-kube-api-access-f77jj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvbkv\" (UID: \"ffb21700-4982-471f-82c5-f5e7a7803d3e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.720124 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.720040    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:35.857251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.857219    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"]
Jul 06 19:58:35.858858 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:58:35.858835    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb21700_4982_471f_82c5_f5e7a7803d3e.slice/crio-247ff82689e62127e8dd932dbb154e47a915a3bab7b88a76c6d76882ef7da89b WatchSource:0}: Error finding container 247ff82689e62127e8dd932dbb154e47a915a3bab7b88a76c6d76882ef7da89b: Status 404 returned error can't find the container with id 247ff82689e62127e8dd932dbb154e47a915a3bab7b88a76c6d76882ef7da89b
Jul 06 19:58:35.860579 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:35.860562    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:58:36.581489 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:36.581462    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:58:36.617846 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:36.617813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv" event={"ID":"ffb21700-4982-471f-82c5-f5e7a7803d3e","Type":"ContainerStarted","Data":"8736a17069cad6e883c05bb3bbddbd4d9fa6a187d05f70403a2af0087c01799d"}
Jul 06 19:58:36.618001 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:36.617868    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv" event={"ID":"ffb21700-4982-471f-82c5-f5e7a7803d3e","Type":"ContainerStarted","Data":"247ff82689e62127e8dd932dbb154e47a915a3bab7b88a76c6d76882ef7da89b"}
Jul 06 19:58:36.638564 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:36.638520    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv" podStartSLOduration=1.468542853 podStartE2EDuration="1.638504786s" totalImagesPullingTime="169.961933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:58:36.637041573 +0000 UTC m=+614.899020449" watchObservedRunningTime="2026-07-06 19:58:36.638504786 +0000 UTC m=+614.900483650"
Jul 06 19:58:44.392459 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:44.392390    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:58:45.721059 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:45.721027    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:45.721537 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:45.721078    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:45.733598 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:45.733577    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:58:46.681763 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:58:46.681736    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvbkv"
Jul 06 19:59:00.599827 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.599795    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"]
Jul 06 19:59:00.606390 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.606359    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.609266 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.609244    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 06 19:59:00.614223 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.614197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"]
Jul 06 19:59:00.746112 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1510425-b50b-4963-9b1c-f353bbc9dc00-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.746279 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w9zx\" (UniqueName: \"kubernetes.io/projected/d1510425-b50b-4963-9b1c-f353bbc9dc00-kube-api-access-5w9zx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.746279 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.746395 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746300    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.746395 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.746506 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.746422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847251 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w9zx\" (UniqueName: \"kubernetes.io/projected/d1510425-b50b-4963-9b1c-f353bbc9dc00-kube-api-access-5w9zx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847290    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847323    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847409 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1510425-b50b-4963-9b1c-f353bbc9dc00-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847697 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847674    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847695    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.847759 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.847732    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.849569 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.849546    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1510425-b50b-4963-9b1c-f353bbc9dc00-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.849735 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.849719    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1510425-b50b-4963-9b1c-f353bbc9dc00-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.855314 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.855266    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w9zx\" (UniqueName: \"kubernetes.io/projected/d1510425-b50b-4963-9b1c-f353bbc9dc00-kube-api-access-5w9zx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kkp8\" (UID: \"d1510425-b50b-4963-9b1c-f353bbc9dc00\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:00.917833 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:00.917812    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:01.054004 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:01.053977    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"]
Jul 06 19:59:01.055497 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:59:01.055468    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1510425_b50b_4963_9b1c_f353bbc9dc00.slice/crio-b54ddecf96f5d4c2cad1931dd543bb2a56ab165322c2ec47b3dea01c42ac90d0 WatchSource:0}: Error finding container b54ddecf96f5d4c2cad1931dd543bb2a56ab165322c2ec47b3dea01c42ac90d0: Status 404 returned error can't find the container with id b54ddecf96f5d4c2cad1931dd543bb2a56ab165322c2ec47b3dea01c42ac90d0
Jul 06 19:59:01.747577 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:01.747487    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8" event={"ID":"d1510425-b50b-4963-9b1c-f353bbc9dc00","Type":"ContainerStarted","Data":"0de1b6c00ccb789d75784f590f9f14bced50a171b21b664adb4aab302b3ec6b6"}
Jul 06 19:59:01.747577 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:01.747536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8" event={"ID":"d1510425-b50b-4963-9b1c-f353bbc9dc00","Type":"ContainerStarted","Data":"b54ddecf96f5d4c2cad1931dd543bb2a56ab165322c2ec47b3dea01c42ac90d0"}
Jul 06 19:59:01.766754 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:01.766710    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8" podStartSLOduration=1.561028544 podStartE2EDuration="1.766696311s" totalImagesPullingTime="205.667767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:01.765106223 +0000 UTC m=+640.027085086" watchObservedRunningTime="2026-07-06 19:59:01.766696311 +0000 UTC m=+640.028675175"
Jul 06 19:59:02.490778 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:02.490738    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:59:07.308657 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.308622    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"]
Jul 06 19:59:07.312444 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.312421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.314776 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.314761    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 06 19:59:07.320903 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.320883    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"]
Jul 06 19:59:07.400701 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.400828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.400828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400739    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljcpg\" (UniqueName: \"kubernetes.io/projected/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-kube-api-access-ljcpg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.400828 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.400988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.400988 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.400885    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502145 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502293 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502293 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502223    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502293 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502250    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljcpg\" (UniqueName: \"kubernetes.io/projected/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-kube-api-access-ljcpg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502408 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502408 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502399    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502568 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502544    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502720 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502701    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.502802 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.502775    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.504350 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.504328    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.504744 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.504728    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.510235 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.510216    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljcpg\" (UniqueName: \"kubernetes.io/projected/a1e7abcd-7cb1-453f-a61a-39bafb6bcb85-kube-api-access-ljcpg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv\" (UID: \"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.622222 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.622152    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:07.761396 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.761368    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"]
Jul 06 19:59:07.763535 ip-10-0-132-142 kubenswrapper[2565]: W0706 19:59:07.763504    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1e7abcd_7cb1_453f_a61a_39bafb6bcb85.slice/crio-9ba16445ae3daad4d764be66d97733aa1085dd726e368eb272737716f8edf1e7 WatchSource:0}: Error finding container 9ba16445ae3daad4d764be66d97733aa1085dd726e368eb272737716f8edf1e7: Status 404 returned error can't find the container with id 9ba16445ae3daad4d764be66d97733aa1085dd726e368eb272737716f8edf1e7
Jul 06 19:59:07.778120 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:07.778095    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv" event={"ID":"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85","Type":"ContainerStarted","Data":"9ba16445ae3daad4d764be66d97733aa1085dd726e368eb272737716f8edf1e7"}
Jul 06 19:59:08.784832 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:08.784789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv" event={"ID":"a1e7abcd-7cb1-453f-a61a-39bafb6bcb85","Type":"ContainerStarted","Data":"6be9d7138cc7046f4b59c19eea8cab8a69d565ce26fe3450398d351507c28fcc"}
Jul 06 19:59:08.804843 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:08.804779    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv" podStartSLOduration=1.61054233 podStartE2EDuration="1.804764367s" totalImagesPullingTime="194.222037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:08.80337096 +0000 UTC m=+647.065349825" watchObservedRunningTime="2026-07-06 19:59:08.804764367 +0000 UTC m=+647.066743270"
Jul 06 19:59:10.890732 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:10.890692    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 19:59:10.918747 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:10.918719    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:10.918887 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:10.918774    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:10.934942 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:10.934894    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:11.811446 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:11.811416    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kkp8"
Jul 06 19:59:17.623239 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:17.623207    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:17.623698 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:17.623254    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:17.635708 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:17.635687    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 19:59:17.842195 ip-10-0-132-142 kubenswrapper[2565]: I0706 19:59:17.842168    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv"
Jul 06 20:00:06.274204 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.274173    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76b84b49d-7qgnt"]
Jul 06 20:00:06.277634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.277615    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.281106 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.281077    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-svfj7\""
Jul 06 20:00:06.281231 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.281113    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 20:00:06.285937 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.285900    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b84b49d-7qgnt"]
Jul 06 20:00:06.384118 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.384085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mlsj\" (UniqueName: \"kubernetes.io/projected/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-kube-api-access-8mlsj\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.384301 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.384143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-tls-cert\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.485498 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.485458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-tls-cert\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.485671 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.485555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mlsj\" (UniqueName: \"kubernetes.io/projected/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-kube-api-access-8mlsj\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.487919 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.487885    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-tls-cert\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.493489 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.493469    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mlsj\" (UniqueName: \"kubernetes.io/projected/026f78c3-544f-4f9f-97bd-5d092ef1c9c9-kube-api-access-8mlsj\") pod \"authorino-76b84b49d-7qgnt\" (UID: \"026f78c3-544f-4f9f-97bd-5d092ef1c9c9\") " pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.588337 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.588256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b84b49d-7qgnt"
Jul 06 20:00:06.733844 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:06.733805    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b84b49d-7qgnt"]
Jul 06 20:00:07.085956 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:07.085894    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b84b49d-7qgnt" event={"ID":"026f78c3-544f-4f9f-97bd-5d092ef1c9c9","Type":"ContainerStarted","Data":"2fbf9333da513ccb9c58ebea8c3d3f3f74d71459d3139a044a33162b4f0f25f0"}
Jul 06 20:00:08.093657 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:08.093622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b84b49d-7qgnt" event={"ID":"026f78c3-544f-4f9f-97bd-5d092ef1c9c9","Type":"ContainerStarted","Data":"870c85f2f589fe04e75bdbf9698eadf182bcb5861d18a520f684f277de00609f"}
Jul 06 20:00:08.116369 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:08.116303    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76b84b49d-7qgnt" podStartSLOduration=1.55907265 podStartE2EDuration="2.116286494s" totalImagesPullingTime="557.213844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:08.114378099 +0000 UTC m=+706.376356964" watchObservedRunningTime="2026-07-06 20:00:08.116286494 +0000 UTC m=+706.378265380"
Jul 06 20:00:35.188444 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:35.188364    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:00:45.085425 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:45.085374    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:00:56.592853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:00:56.592821    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:01:05.399424 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:01:05.399395    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:01:15.985377 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:01:15.985337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:01:45.191205 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:01:45.191116    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:01:56.980655 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:01:56.980617    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:02:31.682130 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:02:31.682095    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:02:41.180039 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:02:41.180004    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:02:49.384537 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:02:49.384505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:02:59.980632 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:02:59.980591    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:03:09.982092 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:09.982050    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:03:21.181340 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:21.181260    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:03:22.627356 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:22.627324    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:03:22.628545 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:22.628520    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:03:32.284772 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:32.284739    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:03:41.380022 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:03:41.379992    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:04:04.585633 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:04:04.585591    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:04:15.588243 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:04:15.588205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:04:39.686201 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:04:39.686155    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:04:49.779137 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:04:49.779061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:05:06.786163 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:05:06.786126    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:05:15.882288 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:05:15.882250    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:05:25.389334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:05:25.389296    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:05:34.790445 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:05:34.790414    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:05:43.185027 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:05:43.184989    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:06:10.414245 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:10.414199    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 20:06:10.414730 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:10.414477    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" podUID="53940790-9a66-455b-886c-02decc68c7c8" containerName="manager" containerID="cri-o://a88ac29e9c0b07cc50d3de5c01023990d991b4520ea0fbce7cdd3980791af4cf" gracePeriod=10
Jul 06 20:06:13.049174 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.049146    2565 generic.go:356] "Generic (PLEG): container finished" podID="53940790-9a66-455b-886c-02decc68c7c8" containerID="a88ac29e9c0b07cc50d3de5c01023990d991b4520ea0fbce7cdd3980791af4cf" exitCode=0
Jul 06 20:06:13.049531 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.049220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" event={"ID":"53940790-9a66-455b-886c-02decc68c7c8","Type":"ContainerDied","Data":"a88ac29e9c0b07cc50d3de5c01023990d991b4520ea0fbce7cdd3980791af4cf"}
Jul 06 20:06:13.167125 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.167102    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 20:06:13.181549 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.181526    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume\") pod \"53940790-9a66-455b-886c-02decc68c7c8\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") "
Jul 06 20:06:13.181658 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.181568    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwmrs\" (UniqueName: \"kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs\") pod \"53940790-9a66-455b-886c-02decc68c7c8\" (UID: \"53940790-9a66-455b-886c-02decc68c7c8\") "
Jul 06 20:06:13.181869 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.181846    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "53940790-9a66-455b-886c-02decc68c7c8" (UID: "53940790-9a66-455b-886c-02decc68c7c8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:06:13.183488 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.183466    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs" (OuterVolumeSpecName: "kube-api-access-bwmrs") pod "53940790-9a66-455b-886c-02decc68c7c8" (UID: "53940790-9a66-455b-886c-02decc68c7c8"). InnerVolumeSpecName "kube-api-access-bwmrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:06:13.282298 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.282258    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53940790-9a66-455b-886c-02decc68c7c8-extensions-socket-volume\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:06:13.282298 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:13.282290    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwmrs\" (UniqueName: \"kubernetes.io/projected/53940790-9a66-455b-886c-02decc68c7c8-kube-api-access-bwmrs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:06:14.055297 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.055229    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"
Jul 06 20:06:14.055773 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.055222    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s" event={"ID":"53940790-9a66-455b-886c-02decc68c7c8","Type":"ContainerDied","Data":"16e8d380bdaca8c2ff7dfc0cafa147a2dbe704b68ca920ab209b38a4c694974d"}
Jul 06 20:06:14.055773 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.055375    2565 scope.go:117] "RemoveContainer" containerID="a88ac29e9c0b07cc50d3de5c01023990d991b4520ea0fbce7cdd3980791af4cf"
Jul 06 20:06:14.077629 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.077605    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 20:06:14.081277 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.081258    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjx7s"]
Jul 06 20:06:14.654722 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:06:14.654684    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53940790-9a66-455b-886c-02decc68c7c8" path="/var/lib/kubelet/pods/53940790-9a66-455b-886c-02decc68c7c8/volumes"
Jul 06 20:07:33.429673 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:07:33.429640    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:07:44.510031 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:07:44.509949    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:07:55.707738 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:07:55.707701    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:04.006081 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:04.006043    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:12.901893 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:12.901861    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:21.102796 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:21.102765    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:22.669099 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:22.669066    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:08:22.670563 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:22.670532    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:08:34.706227 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:34.706177    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:43.896812 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:43.896782    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:08:52.105519 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:08:52.105486    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:09:02.907349 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:09:02.907316    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:09:38.703075 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:09:38.703038    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:09:47.998878 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:09:47.998835    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:09:59.295246 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:09:59.295209    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:06.709352 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:06.709313    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:16.203671 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:16.203630    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:33.001588 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:33.001553    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:40.901293 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:40.901256    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:50.401134 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:50.401053    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:10:59.803195 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:10:59.803158    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:11:09.695530 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:11:09.695493    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:11:20.797325 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:11:20.797286    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:11:33.104241 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:11:33.104207    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:11:42.402565 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:11:42.402529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:11:51.734905 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:11:51.734867    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:03.405926 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:03.405882    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:15.302004 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:15.301920    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:24.419307 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:24.419271    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:34.608165 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:34.608134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:44.603852 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:44.603820    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:12:57.403860 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:12:57.403828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:08.608784 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:08.608739    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:17.495440 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:17.495406    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:22.705584 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:22.705553    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:13:22.709240 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:22.709217    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:13:26.502625 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:26.502587    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:28.094783 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.094744    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:28.095330 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.095311    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53940790-9a66-455b-886c-02decc68c7c8" containerName="manager"
Jul 06 20:13:28.095379 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.095335    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="53940790-9a66-455b-886c-02decc68c7c8" containerName="manager"
Jul 06 20:13:28.095445 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.095433    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="53940790-9a66-455b-886c-02decc68c7c8" containerName="manager"
Jul 06 20:13:28.098781 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.098762    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.101294 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.101275    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:13:28.101405 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.101275    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-81f0cb7a-serving-cert\""
Jul 06 20:13:28.107562 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.107537    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:28.201039 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.201008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwx5w\" (UniqueName: \"kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.201179 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.201079    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.201179 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.201116    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.301971 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.301941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.301971 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.301977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.302196 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.302006    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwx5w\" (UniqueName: \"kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.302682 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.302657    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.304401 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.304373    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.312819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.312798    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwx5w\" (UniqueName: \"kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w\") pod \"maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") " pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.411471 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.411410    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:28.546386 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.546345    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:28.547925 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:13:28.547876    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b9261c_b677_43d1_9e65_dd3832ea1c26.slice/crio-147b528dc24b47b0f344fc6327894459cfc1e8dd8b27ca48e5eab10a93cfdb86 WatchSource:0}: Error finding container 147b528dc24b47b0f344fc6327894459cfc1e8dd8b27ca48e5eab10a93cfdb86: Status 404 returned error can't find the container with id 147b528dc24b47b0f344fc6327894459cfc1e8dd8b27ca48e5eab10a93cfdb86
Jul 06 20:13:28.549228 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:28.549207    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:13:29.364432 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:29.364373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" event={"ID":"d4b9261c-b677-43d1-9e65-dd3832ea1c26","Type":"ContainerStarted","Data":"147b528dc24b47b0f344fc6327894459cfc1e8dd8b27ca48e5eab10a93cfdb86"}
Jul 06 20:13:30.370300 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:30.370255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" event={"ID":"d4b9261c-b677-43d1-9e65-dd3832ea1c26","Type":"ContainerStarted","Data":"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"}
Jul 06 20:13:30.370706 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:30.370330    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:30.393533 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:30.393488    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" podStartSLOduration=0.707240367 podStartE2EDuration="2.393474054s" totalImagesPullingTime="1.686233687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:30.391793797 +0000 UTC m=+1508.653772661" watchObservedRunningTime="2026-07-06 20:13:30.393474054 +0000 UTC m=+1508.655452918"
Jul 06 20:13:33.455565 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.455518    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:33.456068 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.455762    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" podUID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" containerName="maas-api" containerID="cri-o://75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1" gracePeriod=30
Jul 06 20:13:33.701127 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.701102    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:33.748434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.748356    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwx5w\" (UniqueName: \"kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w\") pod \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") "
Jul 06 20:13:33.748434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.748397    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls\") pod \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") "
Jul 06 20:13:33.748653 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.748446    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca\") pod \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\" (UID: \"d4b9261c-b677-43d1-9e65-dd3832ea1c26\") "
Jul 06 20:13:33.748890 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.748857    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4b9261c-b677-43d1-9e65-dd3832ea1c26" (UID: "d4b9261c-b677-43d1-9e65-dd3832ea1c26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:13:33.750584 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.750555    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4b9261c-b677-43d1-9e65-dd3832ea1c26" (UID: "d4b9261c-b677-43d1-9e65-dd3832ea1c26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:13:33.750699 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.750671    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w" (OuterVolumeSpecName: "kube-api-access-rwx5w") pod "d4b9261c-b677-43d1-9e65-dd3832ea1c26" (UID: "d4b9261c-b677-43d1-9e65-dd3832ea1c26"). InnerVolumeSpecName "kube-api-access-rwx5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:13:33.849603 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.849566    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b9261c-b677-43d1-9e65-dd3832ea1c26-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:33.849603 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.849596    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwx5w\" (UniqueName: \"kubernetes.io/projected/d4b9261c-b677-43d1-9e65-dd3832ea1c26-kube-api-access-rwx5w\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:33.849603 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:33.849606    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b9261c-b677-43d1-9e65-dd3832ea1c26-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:34.391890 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.391852    2565 generic.go:356] "Generic (PLEG): container finished" podID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" containerID="75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1" exitCode=0
Jul 06 20:13:34.392093 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.391901    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"
Jul 06 20:13:34.392093 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.391939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" event={"ID":"d4b9261c-b677-43d1-9e65-dd3832ea1c26","Type":"ContainerDied","Data":"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"}
Jul 06 20:13:34.392093 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.391998    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw" event={"ID":"d4b9261c-b677-43d1-9e65-dd3832ea1c26","Type":"ContainerDied","Data":"147b528dc24b47b0f344fc6327894459cfc1e8dd8b27ca48e5eab10a93cfdb86"}
Jul 06 20:13:34.392093 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.392020    2565 scope.go:117] "RemoveContainer" containerID="75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"
Jul 06 20:13:34.401305 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.401286    2565 scope.go:117] "RemoveContainer" containerID="75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"
Jul 06 20:13:34.401561 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:13:34.401541    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1\": container with ID starting with 75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1 not found: ID does not exist" containerID="75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"
Jul 06 20:13:34.401611 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.401569    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1"} err="failed to get container status \"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1\": rpc error: code = NotFound desc = could not find container \"75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1\": container with ID starting with 75940e98f1c59711d914da113f6b3108457ca28ab4969e6db6f41197e31356a1 not found: ID does not exist"
Jul 06 20:13:34.415054 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.415029    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:34.419644 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.419623    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-81f0cb7a-7bb9959599-7sffw"]
Jul 06 20:13:34.655657 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:34.655579    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" path="/var/lib/kubelet/pods/d4b9261c-b677-43d1-9e65-dd3832ea1c26/volumes"
Jul 06 20:13:40.400096 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.400057    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:40.400606 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.400591    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" containerName="maas-api"
Jul 06 20:13:40.400682 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.400612    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" containerName="maas-api"
Jul 06 20:13:40.400735 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.400713    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b9261c-b677-43d1-9e65-dd3832ea1c26" containerName="maas-api"
Jul 06 20:13:40.403837 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.403814    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.406387 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.406364    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-663315aa-serving-cert\""
Jul 06 20:13:40.406524 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.406397    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:13:40.415657 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.415635    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:40.507655 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.507628    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.507819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.507670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.507819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.507694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ghv\" (UniqueName: \"kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.608577 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.608545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ghv\" (UniqueName: \"kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.608721 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.608624    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.608721 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.608652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.609215 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.609193    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.610969 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.610946    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.617077 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.617055    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ghv\" (UniqueName: \"kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv\") pod \"maas-api-e2e-ait-663315aa-64dffd77bf-5jg82\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") " pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.716041 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.715965    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:40.855948 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:40.855484    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:40.860595 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:13:40.860570    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819aa8cf_fe00_4275_8932_a9a13eb41444.slice/crio-352c0d8b8321a83f70d8725d71604216fcf751151cffe64373da57201b8d6565 WatchSource:0}: Error finding container 352c0d8b8321a83f70d8725d71604216fcf751151cffe64373da57201b8d6565: Status 404 returned error can't find the container with id 352c0d8b8321a83f70d8725d71604216fcf751151cffe64373da57201b8d6565
Jul 06 20:13:41.430804 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:41.430772    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" event={"ID":"819aa8cf-fe00-4275-8932-a9a13eb41444","Type":"ContainerStarted","Data":"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"}
Jul 06 20:13:41.431204 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:41.430826    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" event={"ID":"819aa8cf-fe00-4275-8932-a9a13eb41444","Type":"ContainerStarted","Data":"352c0d8b8321a83f70d8725d71604216fcf751151cffe64373da57201b8d6565"}
Jul 06 20:13:41.431204 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:41.430983    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:41.451640 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:41.451593    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" podStartSLOduration=1.107950962 podStartE2EDuration="1.451576986s" totalImagesPullingTime="343.626024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:41.449391895 +0000 UTC m=+1519.711370762" watchObservedRunningTime="2026-07-06 20:13:41.451576986 +0000 UTC m=+1519.713555849"
Jul 06 20:13:44.113352 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:44.113283    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:45.829659 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:45.829628    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:45.830146 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:45.829841    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" podUID="819aa8cf-fe00-4275-8932-a9a13eb41444" containerName="maas-api" containerID="cri-o://50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb" gracePeriod=30
Jul 06 20:13:46.080241 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.080186    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:46.155209 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.155181    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca\") pod \"819aa8cf-fe00-4275-8932-a9a13eb41444\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") "
Jul 06 20:13:46.155331 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.155255    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9ghv\" (UniqueName: \"kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv\") pod \"819aa8cf-fe00-4275-8932-a9a13eb41444\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") "
Jul 06 20:13:46.155386 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.155353    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls\") pod \"819aa8cf-fe00-4275-8932-a9a13eb41444\" (UID: \"819aa8cf-fe00-4275-8932-a9a13eb41444\") "
Jul 06 20:13:46.155599 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.155569    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "819aa8cf-fe00-4275-8932-a9a13eb41444" (UID: "819aa8cf-fe00-4275-8932-a9a13eb41444"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:13:46.157308 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.157286    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "819aa8cf-fe00-4275-8932-a9a13eb41444" (UID: "819aa8cf-fe00-4275-8932-a9a13eb41444"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:13:46.157405 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.157347    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv" (OuterVolumeSpecName: "kube-api-access-x9ghv") pod "819aa8cf-fe00-4275-8932-a9a13eb41444" (UID: "819aa8cf-fe00-4275-8932-a9a13eb41444"). InnerVolumeSpecName "kube-api-access-x9ghv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:13:46.256809 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.256781    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9ghv\" (UniqueName: \"kubernetes.io/projected/819aa8cf-fe00-4275-8932-a9a13eb41444-kube-api-access-x9ghv\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:46.256809 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.256806    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819aa8cf-fe00-4275-8932-a9a13eb41444-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:46.256984 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.256816    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819aa8cf-fe00-4275-8932-a9a13eb41444-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:13:46.456853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.456772    2565 generic.go:356] "Generic (PLEG): container finished" podID="819aa8cf-fe00-4275-8932-a9a13eb41444" containerID="50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb" exitCode=0
Jul 06 20:13:46.456853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.456806    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" event={"ID":"819aa8cf-fe00-4275-8932-a9a13eb41444","Type":"ContainerDied","Data":"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"}
Jul 06 20:13:46.456853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.456816    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"
Jul 06 20:13:46.456853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.456844    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82" event={"ID":"819aa8cf-fe00-4275-8932-a9a13eb41444","Type":"ContainerDied","Data":"352c0d8b8321a83f70d8725d71604216fcf751151cffe64373da57201b8d6565"}
Jul 06 20:13:46.457188 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.456869    2565 scope.go:117] "RemoveContainer" containerID="50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"
Jul 06 20:13:46.466722 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.466704    2565 scope.go:117] "RemoveContainer" containerID="50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"
Jul 06 20:13:46.467094 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:13:46.467075    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb\": container with ID starting with 50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb not found: ID does not exist" containerID="50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"
Jul 06 20:13:46.467168 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.467102    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb"} err="failed to get container status \"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb\": rpc error: code = NotFound desc = could not find container \"50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb\": container with ID starting with 50cd03e2ae6ff644e2c77ee34abe5f90278d9944bc93746313b706c20df67fdb not found: ID does not exist"
Jul 06 20:13:46.483371 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.483338    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:46.485183 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.485159    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-663315aa-64dffd77bf-5jg82"]
Jul 06 20:13:46.655234 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:46.655195    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="819aa8cf-fe00-4275-8932-a9a13eb41444" path="/var/lib/kubelet/pods/819aa8cf-fe00-4275-8932-a9a13eb41444/volumes"
Jul 06 20:13:55.503497 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:55.503464    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:13:58.597478 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.597436    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:13:58.598015 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.597997    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="819aa8cf-fe00-4275-8932-a9a13eb41444" containerName="maas-api"
Jul 06 20:13:58.598079 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.598019    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="819aa8cf-fe00-4275-8932-a9a13eb41444" containerName="maas-api"
Jul 06 20:13:58.598146 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.598134    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="819aa8cf-fe00-4275-8932-a9a13eb41444" containerName="maas-api"
Jul 06 20:13:58.601663 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.601642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.604203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.604178    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2c63cbed-serving-cert\""
Jul 06 20:13:58.604203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.604194    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:13:58.610375 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.610353    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:13:58.754843 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.754813    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.755028 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.754871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b96d7\" (UniqueName: \"kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.755028 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.754936    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.855875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.855797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.855875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.855851    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b96d7\" (UniqueName: \"kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.856097 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.855897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.856097 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:13:58.855940    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-2c63cbed-serving-cert: secret "maas-api-e2e-derive-2c63cbed-serving-cert" not found
Jul 06 20:13:58.856097 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:13:58.856000    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls podName:c15f6a91-a2ce-421c-b20a-cc241e12b26d nodeName:}" failed. No retries permitted until 2026-07-06 20:13:59.355983109 +0000 UTC m=+1537.617961955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls") pod "maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" (UID: "c15f6a91-a2ce-421c-b20a-cc241e12b26d") : secret "maas-api-e2e-derive-2c63cbed-serving-cert" not found
Jul 06 20:13:58.856578 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.856555    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:58.864285 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:58.864263    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b96d7\" (UniqueName: \"kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:59.359651 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:59.359614    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:59.362051 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:59.362021    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") pod \"maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") " pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:59.514020 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:59.513973    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:13:59.658139 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:13:59.658101    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:13:59.662356 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:13:59.662326    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15f6a91_a2ce_421c_b20a_cc241e12b26d.slice/crio-1c84d11da795d49b5aed2c25aa5b02e8108afc7c36e1aff85bc31fce0757fe93 WatchSource:0}: Error finding container 1c84d11da795d49b5aed2c25aa5b02e8108afc7c36e1aff85bc31fce0757fe93: Status 404 returned error can't find the container with id 1c84d11da795d49b5aed2c25aa5b02e8108afc7c36e1aff85bc31fce0757fe93
Jul 06 20:14:00.535257 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:00.535223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" event={"ID":"c15f6a91-a2ce-421c-b20a-cc241e12b26d","Type":"ContainerStarted","Data":"b5f740b7f62dad1409f73dcb160ca8794e5d55fea4148476700e1bf1bdbe347d"}
Jul 06 20:14:00.535404 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:00.535282    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" event={"ID":"c15f6a91-a2ce-421c-b20a-cc241e12b26d","Type":"ContainerStarted","Data":"1c84d11da795d49b5aed2c25aa5b02e8108afc7c36e1aff85bc31fce0757fe93"}
Jul 06 20:14:00.535404 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:00.535385    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:14:00.553596 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:00.553549    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" podStartSLOduration=2.193300928 podStartE2EDuration="2.553536203s" totalImagesPullingTime="360.235275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:00.551266983 +0000 UTC m=+1538.813245847" watchObservedRunningTime="2026-07-06 20:14:00.553536203 +0000 UTC m=+1538.815515067"
Jul 06 20:14:03.415509 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.415474    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:14:03.416001 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.415713    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" podUID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" containerName="maas-api" containerID="cri-o://b5f740b7f62dad1409f73dcb160ca8794e5d55fea4148476700e1bf1bdbe347d" gracePeriod=30
Jul 06 20:14:03.553613 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.553576    2565 generic.go:356] "Generic (PLEG): container finished" podID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" containerID="b5f740b7f62dad1409f73dcb160ca8794e5d55fea4148476700e1bf1bdbe347d" exitCode=0
Jul 06 20:14:03.553774 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.553645    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" event={"ID":"c15f6a91-a2ce-421c-b20a-cc241e12b26d","Type":"ContainerDied","Data":"b5f740b7f62dad1409f73dcb160ca8794e5d55fea4148476700e1bf1bdbe347d"}
Jul 06 20:14:03.666688 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.666628    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:14:03.800953 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.800877    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b96d7\" (UniqueName: \"kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7\") pod \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") "
Jul 06 20:14:03.801120 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.801002    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") pod \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") "
Jul 06 20:14:03.801120 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.801038    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca\") pod \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\" (UID: \"c15f6a91-a2ce-421c-b20a-cc241e12b26d\") "
Jul 06 20:14:03.801435 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.801409    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c15f6a91-a2ce-421c-b20a-cc241e12b26d" (UID: "c15f6a91-a2ce-421c-b20a-cc241e12b26d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:14:03.803039 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.803017    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c15f6a91-a2ce-421c-b20a-cc241e12b26d" (UID: "c15f6a91-a2ce-421c-b20a-cc241e12b26d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:14:03.803163 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.803142    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7" (OuterVolumeSpecName: "kube-api-access-b96d7") pod "c15f6a91-a2ce-421c-b20a-cc241e12b26d" (UID: "c15f6a91-a2ce-421c-b20a-cc241e12b26d"). InnerVolumeSpecName "kube-api-access-b96d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:14:03.902384 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.902350    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c15f6a91-a2ce-421c-b20a-cc241e12b26d-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:14:03.902384 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.902381    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b96d7\" (UniqueName: \"kubernetes.io/projected/c15f6a91-a2ce-421c-b20a-cc241e12b26d-kube-api-access-b96d7\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:14:03.902384 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:03.902392    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c15f6a91-a2ce-421c-b20a-cc241e12b26d-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:14:04.560452 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.560424    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"
Jul 06 20:14:04.560905 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.560447    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl" event={"ID":"c15f6a91-a2ce-421c-b20a-cc241e12b26d","Type":"ContainerDied","Data":"1c84d11da795d49b5aed2c25aa5b02e8108afc7c36e1aff85bc31fce0757fe93"}
Jul 06 20:14:04.560905 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.560509    2565 scope.go:117] "RemoveContainer" containerID="b5f740b7f62dad1409f73dcb160ca8794e5d55fea4148476700e1bf1bdbe347d"
Jul 06 20:14:04.583845 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.583819    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:14:04.586992 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.586965    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2c63cbed-5b7f55f974-dphrl"]
Jul 06 20:14:04.655626 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:14:04.655604    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" path="/var/lib/kubelet/pods/c15f6a91-a2ce-421c-b20a-cc241e12b26d/volumes"
Jul 06 20:15:00.136581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.136545    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722815-fzvwd"]
Jul 06 20:15:00.137065 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.136899    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" containerName="maas-api"
Jul 06 20:15:00.137065 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.136927    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" containerName="maas-api"
Jul 06 20:15:00.137065 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.137009    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15f6a91-a2ce-421c-b20a-cc241e12b26d" containerName="maas-api"
Jul 06 20:15:00.140031 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.140012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:00.142705 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.142678    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:15:00.150053 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.150028    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722815-fzvwd"]
Jul 06 20:15:00.260551 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.260518    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjw95\" (UniqueName: \"kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95\") pod \"maas-api-key-cleanup-29722815-fzvwd\" (UID: \"4892111f-1fb0-45a1-b2dc-401bdbfafe2d\") " pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:00.361806 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.361775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjw95\" (UniqueName: \"kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95\") pod \"maas-api-key-cleanup-29722815-fzvwd\" (UID: \"4892111f-1fb0-45a1-b2dc-401bdbfafe2d\") " pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:00.371819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.371791    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjw95\" (UniqueName: \"kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95\") pod \"maas-api-key-cleanup-29722815-fzvwd\" (UID: \"4892111f-1fb0-45a1-b2dc-401bdbfafe2d\") " pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:00.451755 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.451692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:00.581407 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.581381    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722815-fzvwd"]
Jul 06 20:15:00.583191 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:15:00.583160    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4892111f_1fb0_45a1_b2dc_401bdbfafe2d.slice/crio-d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93 WatchSource:0}: Error finding container d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93: Status 404 returned error can't find the container with id d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93
Jul 06 20:15:00.858438 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:00.858403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd" event={"ID":"4892111f-1fb0-45a1-b2dc-401bdbfafe2d","Type":"ContainerStarted","Data":"d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93"}
Jul 06 20:15:03.877223 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:03.877189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd" event={"ID":"4892111f-1fb0-45a1-b2dc-401bdbfafe2d","Type":"ContainerStarted","Data":"495c0420dbf5895d2cf0031ed09fdb53d1723750154e8241f609f79bb4e184a5"}
Jul 06 20:15:03.893022 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:03.892972    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd" podStartSLOduration=1.076261858 podStartE2EDuration="3.892955976s" totalImagesPullingTime="2.816694118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:03.891862531 +0000 UTC m=+1602.153841397" watchObservedRunningTime="2026-07-06 20:15:03.892955976 +0000 UTC m=+1602.154934839"
Jul 06 20:15:04.883934 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:04.883879    2565 generic.go:356] "Generic (PLEG): container finished" podID="4892111f-1fb0-45a1-b2dc-401bdbfafe2d" containerID="495c0420dbf5895d2cf0031ed09fdb53d1723750154e8241f609f79bb4e184a5" exitCode=0
Jul 06 20:15:04.884315 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:04.883969    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd" event={"ID":"4892111f-1fb0-45a1-b2dc-401bdbfafe2d","Type":"ContainerDied","Data":"495c0420dbf5895d2cf0031ed09fdb53d1723750154e8241f609f79bb4e184a5"}
Jul 06 20:15:06.013200 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.013169    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:06.114007 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.113966    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjw95\" (UniqueName: \"kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95\") pod \"4892111f-1fb0-45a1-b2dc-401bdbfafe2d\" (UID: \"4892111f-1fb0-45a1-b2dc-401bdbfafe2d\") "
Jul 06 20:15:06.116040 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.116013    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95" (OuterVolumeSpecName: "kube-api-access-rjw95") pod "4892111f-1fb0-45a1-b2dc-401bdbfafe2d" (UID: "4892111f-1fb0-45a1-b2dc-401bdbfafe2d"). InnerVolumeSpecName "kube-api-access-rjw95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:15:06.215463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.215360    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjw95\" (UniqueName: \"kubernetes.io/projected/4892111f-1fb0-45a1-b2dc-401bdbfafe2d-kube-api-access-rjw95\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:15:06.895000 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.894966    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd"
Jul 06 20:15:06.895000 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.894980    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722815-fzvwd" event={"ID":"4892111f-1fb0-45a1-b2dc-401bdbfafe2d","Type":"ContainerDied","Data":"d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93"}
Jul 06 20:15:06.895281 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:06.895032    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0afd1ce58c0c5237be0a437e6d43d644436a7345362230e5ae6910083d8de93"
Jul 06 20:15:30.697557 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:30.697467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:15:38.301931 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:38.301864    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:15:47.309088 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.309055    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:15:47.309600 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.309397    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4892111f-1fb0-45a1-b2dc-401bdbfafe2d" containerName="cleanup"
Jul 06 20:15:47.309600 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.309408    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4892111f-1fb0-45a1-b2dc-401bdbfafe2d" containerName="cleanup"
Jul 06 20:15:47.309600 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.309485    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4892111f-1fb0-45a1-b2dc-401bdbfafe2d" containerName="cleanup"
Jul 06 20:15:47.315958 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.315941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.318458 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.318434    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-6b6da4f1-serving-cert\""
Jul 06 20:15:47.318564 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.318472    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:15:47.325850 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.325829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:15:47.447983 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.447941    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.447983 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.447987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8klc2\" (UniqueName: \"kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.448206 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.448035    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.508280 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.508248    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:15:47.549173 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.549136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.549373 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.549349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.549434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.549392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8klc2\" (UniqueName: \"kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.549873 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.549842    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.551800 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.551776    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.557850 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.557825    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8klc2\" (UniqueName: \"kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2\") pod \"maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") " pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.627188 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.627120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:47.760817 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:47.760770    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:15:47.763539 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:15:47.763511    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0303340b_ae32_40a9_bec8_957c9d143c3a.slice/crio-11a8cdcd05f00b0cd4186b1c7065546762bd39eaa002f3eade0fe3317e41d084 WatchSource:0}: Error finding container 11a8cdcd05f00b0cd4186b1c7065546762bd39eaa002f3eade0fe3317e41d084: Status 404 returned error can't find the container with id 11a8cdcd05f00b0cd4186b1c7065546762bd39eaa002f3eade0fe3317e41d084
Jul 06 20:15:48.113562 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:48.113524    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" event={"ID":"0303340b-ae32-40a9-bec8-957c9d143c3a","Type":"ContainerStarted","Data":"11a8cdcd05f00b0cd4186b1c7065546762bd39eaa002f3eade0fe3317e41d084"}
Jul 06 20:15:49.119865 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:49.119834    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" event={"ID":"0303340b-ae32-40a9-bec8-957c9d143c3a","Type":"ContainerStarted","Data":"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"}
Jul 06 20:15:49.120262 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:49.119957    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:49.144101 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:49.144056    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" podStartSLOduration=1.593206407 podStartE2EDuration="2.144041251s" totalImagesPullingTime="550.834844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:49.140353834 +0000 UTC m=+1647.402332697" watchObservedRunningTime="2026-07-06 20:15:49.144041251 +0000 UTC m=+1647.406020114"
Jul 06 20:15:55.130179 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:55.130144    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:58.606365 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:58.606332    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:15:59.333412 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.333378    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:15:59.333649 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.333624    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" podUID="0303340b-ae32-40a9-bec8-957c9d143c3a" containerName="maas-api" containerID="cri-o://ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6" gracePeriod=30
Jul 06 20:15:59.581334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.581311    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:15:59.654559 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.654496    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls\") pod \"0303340b-ae32-40a9-bec8-957c9d143c3a\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") "
Jul 06 20:15:59.654880 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.654562    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8klc2\" (UniqueName: \"kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2\") pod \"0303340b-ae32-40a9-bec8-957c9d143c3a\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") "
Jul 06 20:15:59.654880 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.654615    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca\") pod \"0303340b-ae32-40a9-bec8-957c9d143c3a\" (UID: \"0303340b-ae32-40a9-bec8-957c9d143c3a\") "
Jul 06 20:15:59.655005 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.654985    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0303340b-ae32-40a9-bec8-957c9d143c3a" (UID: "0303340b-ae32-40a9-bec8-957c9d143c3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:15:59.656498 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.656476    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2" (OuterVolumeSpecName: "kube-api-access-8klc2") pod "0303340b-ae32-40a9-bec8-957c9d143c3a" (UID: "0303340b-ae32-40a9-bec8-957c9d143c3a"). InnerVolumeSpecName "kube-api-access-8klc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:15:59.656565 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.656550    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0303340b-ae32-40a9-bec8-957c9d143c3a" (UID: "0303340b-ae32-40a9-bec8-957c9d143c3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:15:59.755804 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.755777    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0303340b-ae32-40a9-bec8-957c9d143c3a-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:15:59.755804 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.755801    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0303340b-ae32-40a9-bec8-957c9d143c3a-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:15:59.755978 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:15:59.755812    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8klc2\" (UniqueName: \"kubernetes.io/projected/0303340b-ae32-40a9-bec8-957c9d143c3a-kube-api-access-8klc2\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:16:00.176524 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.176490    2565 generic.go:356] "Generic (PLEG): container finished" podID="0303340b-ae32-40a9-bec8-957c9d143c3a" containerID="ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6" exitCode=0
Jul 06 20:16:00.176704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.176529    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"
Jul 06 20:16:00.176704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.176578    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" event={"ID":"0303340b-ae32-40a9-bec8-957c9d143c3a","Type":"ContainerDied","Data":"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"}
Jul 06 20:16:00.176704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.176627    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm" event={"ID":"0303340b-ae32-40a9-bec8-957c9d143c3a","Type":"ContainerDied","Data":"11a8cdcd05f00b0cd4186b1c7065546762bd39eaa002f3eade0fe3317e41d084"}
Jul 06 20:16:00.176704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.176652    2565 scope.go:117] "RemoveContainer" containerID="ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"
Jul 06 20:16:00.186288 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.186269    2565 scope.go:117] "RemoveContainer" containerID="ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"
Jul 06 20:16:00.186517 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:16:00.186498    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6\": container with ID starting with ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6 not found: ID does not exist" containerID="ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"
Jul 06 20:16:00.186580 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.186525    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6"} err="failed to get container status \"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6\": rpc error: code = NotFound desc = could not find container \"ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6\": container with ID starting with ab0c48d0467f98144d86080a6f764841d1eb497f116f4486403ca1ec3aeecab6 not found: ID does not exist"
Jul 06 20:16:00.198285 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.198256    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:16:00.203061 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.203040    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-6b6da4f1-cfffbc466-hdlnm"]
Jul 06 20:16:00.655506 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:00.655476    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0303340b-ae32-40a9-bec8-957c9d143c3a" path="/var/lib/kubelet/pods/0303340b-ae32-40a9-bec8-957c9d143c3a/volumes"
Jul 06 20:16:07.603370 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:07.603335    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:16:28.483830 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.483791    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:16:28.484360 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.484343    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0303340b-ae32-40a9-bec8-957c9d143c3a" containerName="maas-api"
Jul 06 20:16:28.484405 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.484363    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0303340b-ae32-40a9-bec8-957c9d143c3a" containerName="maas-api"
Jul 06 20:16:28.484483 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.484466    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0303340b-ae32-40a9-bec8-957c9d143c3a" containerName="maas-api"
Jul 06 20:16:28.487658 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.487638    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.490528 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.490500    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-2bec5f66-serving-cert\""
Jul 06 20:16:28.490706 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.490600    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:16:28.500254 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.500222    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:16:28.594290 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.594261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.594420 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.594327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcffj\" (UniqueName: \"kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.594420 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.594377    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.695486 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.695456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.695608 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.695505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.695608 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.695554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcffj\" (UniqueName: \"kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.695691 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:16:28.695658    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-2bec5f66-serving-cert: secret "maas-api-e2e-mt-2bec5f66-serving-cert" not found
Jul 06 20:16:28.695729 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:16:28.695717    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls podName:109b3afc-46ed-4507-b1b6-813af5ebc162 nodeName:}" failed. No retries permitted until 2026-07-06 20:16:29.195702431 +0000 UTC m=+1687.457681277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls") pod "maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" (UID: "109b3afc-46ed-4507-b1b6-813af5ebc162") : secret "maas-api-e2e-mt-2bec5f66-serving-cert" not found
Jul 06 20:16:28.696103 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.696083    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:28.704240 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:28.704214    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcffj\" (UniqueName: \"kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:29.199708 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:29.199671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:29.202067 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:29.202048    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") pod \"maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") " pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:29.406995 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:29.406960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:29.538500 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:29.538474    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:16:29.540024 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:16:29.539996    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109b3afc_46ed_4507_b1b6_813af5ebc162.slice/crio-fa1c6584384a5987bb37973b57472e30262e99b6a5035c73588b989f4306dab8 WatchSource:0}: Error finding container fa1c6584384a5987bb37973b57472e30262e99b6a5035c73588b989f4306dab8: Status 404 returned error can't find the container with id fa1c6584384a5987bb37973b57472e30262e99b6a5035c73588b989f4306dab8
Jul 06 20:16:30.339444 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:30.339410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" event={"ID":"109b3afc-46ed-4507-b1b6-813af5ebc162","Type":"ContainerStarted","Data":"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"}
Jul 06 20:16:30.339643 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:30.339461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" event={"ID":"109b3afc-46ed-4507-b1b6-813af5ebc162","Type":"ContainerStarted","Data":"fa1c6584384a5987bb37973b57472e30262e99b6a5035c73588b989f4306dab8"}
Jul 06 20:16:30.339643 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:30.339511    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:30.357577 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:30.357523    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" podStartSLOduration=1.939925991 podStartE2EDuration="2.357509319s" totalImagesPullingTime="417.583328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:30.356348064 +0000 UTC m=+1688.618326928" watchObservedRunningTime="2026-07-06 20:16:30.357509319 +0000 UTC m=+1688.619488198"
Jul 06 20:16:33.224202 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:33.224162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:16:36.349442 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:36.349416    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:16:42.202491 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.202458    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:16:42.206244 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.206211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.209738 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.209709    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2bec5f66\"/\"kube-root-ca.crt\""
Jul 06 20:16:42.209868 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.209761    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2bec5f66\"/\"default-dockercfg-nmn42\""
Jul 06 20:16:42.209868 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.209832    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2bec5f66\"/\"openshift-service-ca.crt\""
Jul 06 20:16:42.210047 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.209874    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2bec5f66\"/\"e2e-lifecycle-model-2bec5f66-kserve-self-signed-certs\""
Jul 06 20:16:42.216009 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.215986    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:16:42.317363 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317332    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.317501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxp8q\" (UniqueName: \"kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.317501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.317501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.317620 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.317620 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.317539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418099 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418110    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418191    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxp8q\" (UniqueName: \"kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418500 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418445    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418571 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418501    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.418658 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.418638    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.420357 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.420335    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.420665 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.420645    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.425902 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.425879    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxp8q\" (UniqueName: \"kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q\") pod \"e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") " pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.516016 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.515996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:42.649548 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:42.649519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:16:42.652829 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:16:42.652799    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a4977dc_f3c8_4e53_965c_364cc5cf6130.slice/crio-0fceefa9b6ec7b2ad4cb90863f5517822496b34995871083a05f91a07b5a1ea8 WatchSource:0}: Error finding container 0fceefa9b6ec7b2ad4cb90863f5517822496b34995871083a05f91a07b5a1ea8: Status 404 returned error can't find the container with id 0fceefa9b6ec7b2ad4cb90863f5517822496b34995871083a05f91a07b5a1ea8
Jul 06 20:16:43.417959 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:43.417902    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" event={"ID":"7a4977dc-f3c8-4e53-965c-364cc5cf6130","Type":"ContainerStarted","Data":"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"}
Jul 06 20:16:43.417959 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:43.417966    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" event={"ID":"7a4977dc-f3c8-4e53-965c-364cc5cf6130","Type":"ContainerStarted","Data":"0fceefa9b6ec7b2ad4cb90863f5517822496b34995871083a05f91a07b5a1ea8"}
Jul 06 20:16:43.435588 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:43.435537    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" podStartSLOduration=1.435523029 podStartE2EDuration="1.435523029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:43.434591618 +0000 UTC m=+1701.696570483" watchObservedRunningTime="2026-07-06 20:16:43.435523029 +0000 UTC m=+1701.697501893"
Jul 06 20:16:45.898503 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:45.898426    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:16:52.516245 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:52.516208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:52.516624 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:52.516260    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:52.528466 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:52.528441    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:53.481631 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:53.481603    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:16:57.900333 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:16:57.900291    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:17:00.738969 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:00.738940    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:17:00.739485 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:00.739151    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" podUID="109b3afc-46ed-4507-b1b6-813af5ebc162" containerName="maas-api" containerID="cri-o://e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c" gracePeriod=30
Jul 06 20:17:00.978836 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:00.978813    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:17:01.075646 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.075610    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") pod \"109b3afc-46ed-4507-b1b6-813af5ebc162\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") "
Jul 06 20:17:01.075791 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.075658    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca\") pod \"109b3afc-46ed-4507-b1b6-813af5ebc162\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") "
Jul 06 20:17:01.075791 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.075686    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcffj\" (UniqueName: \"kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj\") pod \"109b3afc-46ed-4507-b1b6-813af5ebc162\" (UID: \"109b3afc-46ed-4507-b1b6-813af5ebc162\") "
Jul 06 20:17:01.076074 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.076037    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "109b3afc-46ed-4507-b1b6-813af5ebc162" (UID: "109b3afc-46ed-4507-b1b6-813af5ebc162"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:17:01.077650 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.077628    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "109b3afc-46ed-4507-b1b6-813af5ebc162" (UID: "109b3afc-46ed-4507-b1b6-813af5ebc162"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:17:01.077749 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.077670    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj" (OuterVolumeSpecName: "kube-api-access-hcffj") pod "109b3afc-46ed-4507-b1b6-813af5ebc162" (UID: "109b3afc-46ed-4507-b1b6-813af5ebc162"). InnerVolumeSpecName "kube-api-access-hcffj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:17:01.176797 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.176769    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/109b3afc-46ed-4507-b1b6-813af5ebc162-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:01.176797 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.176794    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/109b3afc-46ed-4507-b1b6-813af5ebc162-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:01.176990 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.176806    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcffj\" (UniqueName: \"kubernetes.io/projected/109b3afc-46ed-4507-b1b6-813af5ebc162-kube-api-access-hcffj\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:01.514963 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.514927    2565 generic.go:356] "Generic (PLEG): container finished" podID="109b3afc-46ed-4507-b1b6-813af5ebc162" containerID="e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c" exitCode=0
Jul 06 20:17:01.515117 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.514985    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"
Jul 06 20:17:01.515117 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.515002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" event={"ID":"109b3afc-46ed-4507-b1b6-813af5ebc162","Type":"ContainerDied","Data":"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"}
Jul 06 20:17:01.515117 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.515056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd" event={"ID":"109b3afc-46ed-4507-b1b6-813af5ebc162","Type":"ContainerDied","Data":"fa1c6584384a5987bb37973b57472e30262e99b6a5035c73588b989f4306dab8"}
Jul 06 20:17:01.515117 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.515079    2565 scope.go:117] "RemoveContainer" containerID="e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"
Jul 06 20:17:01.524515 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.524498    2565 scope.go:117] "RemoveContainer" containerID="e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"
Jul 06 20:17:01.524775 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:17:01.524757    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c\": container with ID starting with e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c not found: ID does not exist" containerID="e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"
Jul 06 20:17:01.524839 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.524783    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c"} err="failed to get container status \"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c\": rpc error: code = NotFound desc = could not find container \"e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c\": container with ID starting with e7773d36c42130749da002675047cf21aa31937f84635bbd148e1b99db59830c not found: ID does not exist"
Jul 06 20:17:01.536513 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.536488    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:17:01.539753 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:01.539732    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-2bec5f66-57bc6c6f57-zh9gd"]
Jul 06 20:17:02.655363 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:02.655329    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="109b3afc-46ed-4507-b1b6-813af5ebc162" path="/var/lib/kubelet/pods/109b3afc-46ed-4507-b1b6-813af5ebc162/volumes"
Jul 06 20:17:11.200607 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:11.200575    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:17:11.582581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:11.582551    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:17:26.602595 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.602545    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" podUID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" containerName="main" containerID="cri-o://e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c" gracePeriod=45
Jul 06 20:17:26.852068 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.852045    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:17:26.887434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887367    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxp8q\" (UniqueName: \"kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887406    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887451    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887477    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887520    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887554    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm\") pod \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\" (UID: \"7a4977dc-f3c8-4e53-965c-364cc5cf6130\") "
Jul 06 20:17:26.887848 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887655    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home" (OuterVolumeSpecName: "home") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:17:26.887848 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887721    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:17:26.888003 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887861    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:26.888003 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887869    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:17:26.888003 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.887878    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:26.889782 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.889745    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q" (OuterVolumeSpecName: "kube-api-access-bxp8q") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "kube-api-access-bxp8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:17:26.889895 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.889834    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:17:26.890159 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.890134    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm" (OuterVolumeSpecName: "dshm") pod "7a4977dc-f3c8-4e53-965c-364cc5cf6130" (UID: "7a4977dc-f3c8-4e53-965c-364cc5cf6130"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:17:26.988778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.988753    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:26.988778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.988775    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4977dc-f3c8-4e53-965c-364cc5cf6130-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:26.988962 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.988785    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxp8q\" (UniqueName: \"kubernetes.io/projected/7a4977dc-f3c8-4e53-965c-364cc5cf6130-kube-api-access-bxp8q\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:26.988962 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:26.988795    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4977dc-f3c8-4e53-965c-364cc5cf6130-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:17:27.657090 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.657061    2565 generic.go:356] "Generic (PLEG): container finished" podID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" containerID="e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c" exitCode=0
Jul 06 20:17:27.657505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.657097    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"
Jul 06 20:17:27.657505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.657105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" event={"ID":"7a4977dc-f3c8-4e53-965c-364cc5cf6130","Type":"ContainerDied","Data":"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"}
Jul 06 20:17:27.657505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.657131    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc" event={"ID":"7a4977dc-f3c8-4e53-965c-364cc5cf6130","Type":"ContainerDied","Data":"0fceefa9b6ec7b2ad4cb90863f5517822496b34995871083a05f91a07b5a1ea8"}
Jul 06 20:17:27.657505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.657147    2565 scope.go:117] "RemoveContainer" containerID="e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"
Jul 06 20:17:27.666322 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.666306    2565 scope.go:117] "RemoveContainer" containerID="e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"
Jul 06 20:17:27.666578 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:17:27.666562    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c\": container with ID starting with e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c not found: ID does not exist" containerID="e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"
Jul 06 20:17:27.666624 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.666585    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c"} err="failed to get container status \"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c\": rpc error: code = NotFound desc = could not find container \"e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c\": container with ID starting with e7edfa38fe5fe46eb92ac22b022ba9ba6f8faf6b1817d6c51cba911d3bc25c1c not found: ID does not exist"
Jul 06 20:17:27.677894 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.677873    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:17:27.686621 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:27.684068    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2bec5f66/e2e-lifecycle-model-2bec5f66-kserve-67c86d7-p5mtc"]
Jul 06 20:17:28.655395 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:28.655361    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" path="/var/lib/kubelet/pods/7a4977dc-f3c8-4e53-965c-364cc5cf6130/volumes"
Jul 06 20:17:51.299334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:17:51.299297    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:18:00.311423 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:00.311383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:18:10.507930 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:10.507876    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:18:22.739102 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:22.739077    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:18:22.744671 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:22.744652    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:18:53.418481 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418448    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418822    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="109b3afc-46ed-4507-b1b6-813af5ebc162" containerName="maas-api"
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418835    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="109b3afc-46ed-4507-b1b6-813af5ebc162" containerName="maas-api"
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418845    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" containerName="main"
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418851    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" containerName="main"
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418906    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4977dc-f3c8-4e53-965c-364cc5cf6130" containerName="main"
Jul 06 20:18:53.418960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.418931    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="109b3afc-46ed-4507-b1b6-813af5ebc162" containerName="maas-api"
Jul 06 20:18:53.421716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.421699    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.424302 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.424279    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-e7dc01-serving-cert\""
Jul 06 20:18:53.424422 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.424371    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:18:53.435621 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.435598    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:18:53.479098 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.479071    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwfc\" (UniqueName: \"kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.479221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.479184    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.479221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.479207    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.579761 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.579732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwfc\" (UniqueName: \"kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.579887 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.579801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.579887 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.579820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.580042 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:18:53.579903    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-e7dc01-serving-cert: secret "maas-api-e2e-inf-a-e7dc01-serving-cert" not found
Jul 06 20:18:53.580042 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:18:53.579987    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls podName:de484a37-e521-4db3-9890-c1161f97bf81 nodeName:}" failed. No retries permitted until 2026-07-06 20:18:54.079971776 +0000 UTC m=+1832.341950621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls") pod "maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" (UID: "de484a37-e521-4db3-9890-c1161f97bf81") : secret "maas-api-e2e-inf-a-e7dc01-serving-cert" not found
Jul 06 20:18:53.580365 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.580346    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:53.591545 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:53.591523    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwfc\" (UniqueName: \"kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:54.083750 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:54.083719    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:54.086062 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:54.086041    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") " pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:54.332569 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:54.332534    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:54.463021 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:54.462992    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:18:54.464681 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:18:54.464657    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde484a37_e521_4db3_9890_c1161f97bf81.slice/crio-459f2cf0bcc099c8802b6c86414f1d0abe03ae495009b380d8beb2fa3f07bd25 WatchSource:0}: Error finding container 459f2cf0bcc099c8802b6c86414f1d0abe03ae495009b380d8beb2fa3f07bd25: Status 404 returned error can't find the container with id 459f2cf0bcc099c8802b6c86414f1d0abe03ae495009b380d8beb2fa3f07bd25
Jul 06 20:18:54.466470 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:54.466452    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:18:55.102986 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:55.102953    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" event={"ID":"de484a37-e521-4db3-9890-c1161f97bf81","Type":"ContainerStarted","Data":"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"}
Jul 06 20:18:55.103113 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:55.103010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" event={"ID":"de484a37-e521-4db3-9890-c1161f97bf81","Type":"ContainerStarted","Data":"459f2cf0bcc099c8802b6c86414f1d0abe03ae495009b380d8beb2fa3f07bd25"}
Jul 06 20:18:55.103113 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:55.103059    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:18:55.120906 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:18:55.120859    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" podStartSLOduration=1.60439507 podStartE2EDuration="2.120845946s" totalImagesPullingTime="516.450876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:55.118236456 +0000 UTC m=+1833.380215321" watchObservedRunningTime="2026-07-06 20:18:55.120845946 +0000 UTC m=+1833.382824810"
Jul 06 20:19:00.507313 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:00.507273    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:19:01.114500 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.114473    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:19:01.271112 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.271079    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:19:01.274883 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.274856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.277147 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.277127    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-ec5878-serving-cert\""
Jul 06 20:19:01.290032 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.290006    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:19:01.338963 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.338930    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfkxb\" (UniqueName: \"kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.339067 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.338997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.339067 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.339016    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.439505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.439443    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.439505 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.439476    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.439663 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.439517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkxb\" (UniqueName: \"kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.439663 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:19:01.439631    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-ec5878-serving-cert: secret "maas-api-e2e-inf-b-ec5878-serving-cert" not found
Jul 06 20:19:01.439780 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:19:01.439702    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls podName:11b5fa46-8723-4a17-94cb-86a2cf4aa1df nodeName:}" failed. No retries permitted until 2026-07-06 20:19:01.939686135 +0000 UTC m=+1840.201664977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls") pod "maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" (UID: "11b5fa46-8723-4a17-94cb-86a2cf4aa1df") : secret "maas-api-e2e-inf-b-ec5878-serving-cert" not found
Jul 06 20:19:01.440200 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.440177    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.448696 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.448673    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfkxb\" (UniqueName: \"kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.945383 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.945343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:01.947704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:01.947684    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") pod \"maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") " pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:02.187894 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:02.187855    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:02.319828 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:02.319804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:19:02.321530 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:19:02.321493    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b5fa46_8723_4a17_94cb_86a2cf4aa1df.slice/crio-964aaf74c53a9d2afe6c8573bef29c85f628096dd65ffb009c42bd8f563579ba WatchSource:0}: Error finding container 964aaf74c53a9d2afe6c8573bef29c85f628096dd65ffb009c42bd8f563579ba: Status 404 returned error can't find the container with id 964aaf74c53a9d2afe6c8573bef29c85f628096dd65ffb009c42bd8f563579ba
Jul 06 20:19:03.146862 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:03.146832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" event={"ID":"11b5fa46-8723-4a17-94cb-86a2cf4aa1df","Type":"ContainerStarted","Data":"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"}
Jul 06 20:19:03.147279 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:03.146880    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" event={"ID":"11b5fa46-8723-4a17-94cb-86a2cf4aa1df","Type":"ContainerStarted","Data":"964aaf74c53a9d2afe6c8573bef29c85f628096dd65ffb009c42bd8f563579ba"}
Jul 06 20:19:03.147279 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:03.146941    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:03.166486 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:03.166438    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" podStartSLOduration=1.796381226 podStartE2EDuration="2.166422663s" totalImagesPullingTime="370.041437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:03.165058706 +0000 UTC m=+1841.427037569" watchObservedRunningTime="2026-07-06 20:19:03.166422663 +0000 UTC m=+1841.428401554"
Jul 06 20:19:09.157768 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:09.157741    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:19:14.298774 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:14.298738    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:19:24.004861 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.004825    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:19:24.008532 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.008515    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.012637 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.012613    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7dc01\"/\"openshift-service-ca.crt\""
Jul 06 20:19:24.012637 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.012628    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7dc01\"/\"default-dockercfg-wfkqq\""
Jul 06 20:19:24.012783 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.012669    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7dc01\"/\"kube-root-ca.crt\""
Jul 06 20:19:24.012783 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.012678    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7dc01\"/\"test-model-e7dc01-kserve-self-signed-certs\""
Jul 06 20:19:24.016853 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.016829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:19:24.137390 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.137559 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.137559 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.137559 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.137559 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9dh\" (UniqueName: \"kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.137719 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.137580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238371 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238524 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238524 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238429    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238524 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9dh\" (UniqueName: \"kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238662 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238662 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238789 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238759    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238848 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238829    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.238901 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.238869    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.240844 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.240818    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.241026 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.241007    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.246379 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.246357    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9dh\" (UniqueName: \"kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh\") pod \"test-model-e7dc01-kserve-74d87c8469-59cf9\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") " pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.318099 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.318038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:24.450999 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:24.450973    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:19:24.453046 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:19:24.453019    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc00f50f3_8cf2_4232_ba49_a49af4c603f2.slice/crio-fafc57625a92f22a93847d253d02f05f6a21fb6346b528017895e0616397e182 WatchSource:0}: Error finding container fafc57625a92f22a93847d253d02f05f6a21fb6346b528017895e0616397e182: Status 404 returned error can't find the container with id fafc57625a92f22a93847d253d02f05f6a21fb6346b528017895e0616397e182
Jul 06 20:19:25.263196 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:25.263157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" event={"ID":"c00f50f3-8cf2-4232-ba49-a49af4c603f2","Type":"ContainerStarted","Data":"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"}
Jul 06 20:19:25.263551 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:25.263206    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" event={"ID":"c00f50f3-8cf2-4232-ba49-a49af4c603f2","Type":"ContainerStarted","Data":"fafc57625a92f22a93847d253d02f05f6a21fb6346b528017895e0616397e182"}
Jul 06 20:19:25.279420 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:25.279377    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" podStartSLOduration=2.279361931 podStartE2EDuration="2.279361931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:25.278843682 +0000 UTC m=+1863.540822559" watchObservedRunningTime="2026-07-06 20:19:25.279361931 +0000 UTC m=+1863.541340795"
Jul 06 20:19:27.203287 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:27.203249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:19:34.318137 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:34.318105    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:34.318137 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:34.318145    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:34.330714 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:34.330689    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:35.331257 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:35.331227    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:19:38.002357 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:38.002320    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:19:42.109650 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.109616    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:19:42.113401 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.113383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.117302 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.117277    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ec5878\"/\"test-model-ec5878-kserve-self-signed-certs\""
Jul 06 20:19:42.117442 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.117327    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ec5878\"/\"openshift-service-ca.crt\""
Jul 06 20:19:42.117442 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.117400    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ec5878\"/\"kube-root-ca.crt\""
Jul 06 20:19:42.117442 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.117278    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ec5878\"/\"default-dockercfg-4qdzz\""
Jul 06 20:19:42.123540 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.123519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:19:42.183486 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183454    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.183486 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.183685 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183507    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.183685 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183614    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.183685 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24h7g\" (UniqueName: \"kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.183685 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.183683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284485 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284485 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284549    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284585    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24h7g\" (UniqueName: \"kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284602    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.284957 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284877    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.285021 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284975    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.285021 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.284999    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.286673 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.286656    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.287058 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.287040    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.293688 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.293666    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24h7g\" (UniqueName: \"kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g\") pod \"test-model-ec5878-kserve-7d6748bb49-ndlpl\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") " pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.424245 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.424184    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:42.558429 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:42.558399    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:19:42.559709 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:19:42.559671    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c995d26_042b_4c93_8d70_a59fe8f2b98a.slice/crio-6ae55baad87850d81d178cc4cb3b4aa3cee0aa12fc71cdd1fcfdb43d95451d10 WatchSource:0}: Error finding container 6ae55baad87850d81d178cc4cb3b4aa3cee0aa12fc71cdd1fcfdb43d95451d10: Status 404 returned error can't find the container with id 6ae55baad87850d81d178cc4cb3b4aa3cee0aa12fc71cdd1fcfdb43d95451d10
Jul 06 20:19:43.363010 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:43.362972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" event={"ID":"2c995d26-042b-4c93-8d70-a59fe8f2b98a","Type":"ContainerStarted","Data":"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"}
Jul 06 20:19:43.363382 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:43.363019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" event={"ID":"2c995d26-042b-4c93-8d70-a59fe8f2b98a","Type":"ContainerStarted","Data":"6ae55baad87850d81d178cc4cb3b4aa3cee0aa12fc71cdd1fcfdb43d95451d10"}
Jul 06 20:19:43.385698 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:43.385651    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" podStartSLOduration=1.385637795 podStartE2EDuration="1.385637795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:43.384369771 +0000 UTC m=+1881.646348638" watchObservedRunningTime="2026-07-06 20:19:43.385637795 +0000 UTC m=+1881.647616659"
Jul 06 20:19:48.397879 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:48.397798    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:19:52.425081 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:52.425046    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:52.425571 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:52.425101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:52.437046 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:52.437014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:19:53.425259 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:19:53.425229    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:20:24.692661 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:24.692628    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:20:25.696553 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:25.696519    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:20:26.503930 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.503883    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:20:26.504119 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.504096    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" podUID="de484a37-e521-4db3-9890-c1161f97bf81" containerName="maas-api" containerID="cri-o://58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6" gracePeriod=30
Jul 06 20:20:26.946746 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.946723    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:20:26.966335 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.966307    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") pod \"de484a37-e521-4db3-9890-c1161f97bf81\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") "
Jul 06 20:20:26.966494 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.966359    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpwfc\" (UniqueName: \"kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc\") pod \"de484a37-e521-4db3-9890-c1161f97bf81\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") "
Jul 06 20:20:26.966494 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.966471    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca\") pod \"de484a37-e521-4db3-9890-c1161f97bf81\" (UID: \"de484a37-e521-4db3-9890-c1161f97bf81\") "
Jul 06 20:20:26.966966 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.966901    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de484a37-e521-4db3-9890-c1161f97bf81" (UID: "de484a37-e521-4db3-9890-c1161f97bf81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:20:26.968596 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.968574    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc" (OuterVolumeSpecName: "kube-api-access-kpwfc") pod "de484a37-e521-4db3-9890-c1161f97bf81" (UID: "de484a37-e521-4db3-9890-c1161f97bf81"). InnerVolumeSpecName "kube-api-access-kpwfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:20:26.968992 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:26.968969    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de484a37-e521-4db3-9890-c1161f97bf81" (UID: "de484a37-e521-4db3-9890-c1161f97bf81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:20:27.067536 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.067475    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de484a37-e521-4db3-9890-c1161f97bf81-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:27.067536 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.067499    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de484a37-e521-4db3-9890-c1161f97bf81-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:27.067536 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.067511    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpwfc\" (UniqueName: \"kubernetes.io/projected/de484a37-e521-4db3-9890-c1161f97bf81-kube-api-access-kpwfc\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:27.597669 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.597640    2565 generic.go:356] "Generic (PLEG): container finished" podID="de484a37-e521-4db3-9890-c1161f97bf81" containerID="58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6" exitCode=0
Jul 06 20:20:27.597843 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.597682    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"
Jul 06 20:20:27.597843 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.597726    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" event={"ID":"de484a37-e521-4db3-9890-c1161f97bf81","Type":"ContainerDied","Data":"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"}
Jul 06 20:20:27.597843 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.597776    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5" event={"ID":"de484a37-e521-4db3-9890-c1161f97bf81","Type":"ContainerDied","Data":"459f2cf0bcc099c8802b6c86414f1d0abe03ae495009b380d8beb2fa3f07bd25"}
Jul 06 20:20:27.597843 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.597799    2565 scope.go:117] "RemoveContainer" containerID="58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"
Jul 06 20:20:27.607058 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.607040    2565 scope.go:117] "RemoveContainer" containerID="58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"
Jul 06 20:20:27.607284 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:20:27.607269    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6\": container with ID starting with 58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6 not found: ID does not exist" containerID="58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"
Jul 06 20:20:27.607346 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.607291    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6"} err="failed to get container status \"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6\": rpc error: code = NotFound desc = could not find container \"58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6\": container with ID starting with 58e5f5b1b4962bf3a13d7f2bdb408769ce25a863a28c7157e5b314174370ffe6 not found: ID does not exist"
Jul 06 20:20:27.619879 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.619850    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:20:27.623128 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:27.623106    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-e7dc01-76f559b5fd-skjx5"]
Jul 06 20:20:28.654850 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:28.654818    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de484a37-e521-4db3-9890-c1161f97bf81" path="/var/lib/kubelet/pods/de484a37-e521-4db3-9890-c1161f97bf81/volumes"
Jul 06 20:20:29.602552 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:29.602517    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:20:39.712160 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.712123    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" podUID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" containerName="main" containerID="cri-o://9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b" gracePeriod=45
Jul 06 20:20:39.954960 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.954936    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:20:39.979438 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979385    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979438 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979422    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979476    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979526    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979563    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979745 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979597    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9dh\" (UniqueName: \"kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh\") pod \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\" (UID: \"c00f50f3-8cf2-4232-ba49-a49af4c603f2\") "
Jul 06 20:20:39.979745 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979633    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home" (OuterVolumeSpecName: "home") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:39.979745 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979724    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:39.979892 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979811    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:39.979984 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979965    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:39.980038 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.979988    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:39.980038 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.980000    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:39.981509 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.981481    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm" (OuterVolumeSpecName: "dshm") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:39.982011 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.981987    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh" (OuterVolumeSpecName: "kube-api-access-rx9dh") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "kube-api-access-rx9dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:20:39.982011 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:39.981997    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c00f50f3-8cf2-4232-ba49-a49af4c603f2" (UID: "c00f50f3-8cf2-4232-ba49-a49af4c603f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:20:40.081237 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.081212    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00f50f3-8cf2-4232-ba49-a49af4c603f2-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:40.081237 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.081235    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f50f3-8cf2-4232-ba49-a49af4c603f2-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:40.081375 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.081245    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9dh\" (UniqueName: \"kubernetes.io/projected/c00f50f3-8cf2-4232-ba49-a49af4c603f2-kube-api-access-rx9dh\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:40.667335 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.667304    2565 generic.go:356] "Generic (PLEG): container finished" podID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" containerID="9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b" exitCode=0
Jul 06 20:20:40.667495 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.667357    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"
Jul 06 20:20:40.667495 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.667365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" event={"ID":"c00f50f3-8cf2-4232-ba49-a49af4c603f2","Type":"ContainerDied","Data":"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"}
Jul 06 20:20:40.667495 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.667401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9" event={"ID":"c00f50f3-8cf2-4232-ba49-a49af4c603f2","Type":"ContainerDied","Data":"fafc57625a92f22a93847d253d02f05f6a21fb6346b528017895e0616397e182"}
Jul 06 20:20:40.667495 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.667423    2565 scope.go:117] "RemoveContainer" containerID="9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"
Jul 06 20:20:40.676069 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.676049    2565 scope.go:117] "RemoveContainer" containerID="9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"
Jul 06 20:20:40.676321 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:20:40.676302    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b\": container with ID starting with 9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b not found: ID does not exist" containerID="9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"
Jul 06 20:20:40.676384 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.676332    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b"} err="failed to get container status \"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b\": rpc error: code = NotFound desc = could not find container \"9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b\": container with ID starting with 9ee5eeab6c981d984e552fe8f327941c7f92dd3530b21da5973885bacb79cb5b not found: ID does not exist"
Jul 06 20:20:40.684167 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.684144    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:20:40.687636 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.687615    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e7dc01/test-model-e7dc01-kserve-74d87c8469-59cf9"]
Jul 06 20:20:40.716760 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.716735    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" podUID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" containerName="main" containerID="cri-o://2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c" gracePeriod=45
Jul 06 20:20:40.960784 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.960762    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:20:40.989745 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989721    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.989924 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989756    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24h7g\" (UniqueName: \"kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.989924 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989782    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.989924 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989817    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.989924 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989854    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.989924 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989883    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm\") pod \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\" (UID: \"2c995d26-042b-4c93-8d70-a59fe8f2b98a\") "
Jul 06 20:20:40.990175 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.989984    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:40.990175 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.990145    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:40.990175 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.990159    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home" (OuterVolumeSpecName: "home") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:40.990323 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.990207    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:40.991943 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.991896    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g" (OuterVolumeSpecName: "kube-api-access-24h7g") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "kube-api-access-24h7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:20:40.992055 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.991988    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:20:40.992110 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:40.992066    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm" (OuterVolumeSpecName: "dshm") pod "2c995d26-042b-4c93-8d70-a59fe8f2b98a" (UID: "2c995d26-042b-4c93-8d70-a59fe8f2b98a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:20:41.091142 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.091103    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:41.091142 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.091138    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:41.091142 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.091149    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24h7g\" (UniqueName: \"kubernetes.io/projected/2c995d26-042b-4c93-8d70-a59fe8f2b98a-kube-api-access-24h7g\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:41.091334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.091158    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:41.091334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.091187    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c995d26-042b-4c93-8d70-a59fe8f2b98a-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:20:41.674235 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.674205    2565 generic.go:356] "Generic (PLEG): container finished" podID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" containerID="2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c" exitCode=0
Jul 06 20:20:41.674406 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.674239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" event={"ID":"2c995d26-042b-4c93-8d70-a59fe8f2b98a","Type":"ContainerDied","Data":"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"}
Jul 06 20:20:41.674406 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.674242    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"
Jul 06 20:20:41.674406 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.674281    2565 scope.go:117] "RemoveContainer" containerID="2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"
Jul 06 20:20:41.674406 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.674271    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl" event={"ID":"2c995d26-042b-4c93-8d70-a59fe8f2b98a","Type":"ContainerDied","Data":"6ae55baad87850d81d178cc4cb3b4aa3cee0aa12fc71cdd1fcfdb43d95451d10"}
Jul 06 20:20:41.683383 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.683368    2565 scope.go:117] "RemoveContainer" containerID="2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"
Jul 06 20:20:41.683639 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:20:41.683618    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c\": container with ID starting with 2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c not found: ID does not exist" containerID="2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"
Jul 06 20:20:41.683705 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.683645    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c"} err="failed to get container status \"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c\": rpc error: code = NotFound desc = could not find container \"2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c\": container with ID starting with 2cb18011502f92f16adfb637c5b293e4dd9d518e42b46d6f192849e967bbf85c not found: ID does not exist"
Jul 06 20:20:41.695694 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.695667    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:20:41.702261 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:41.702237    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ec5878/test-model-ec5878-kserve-7d6748bb49-ndlpl"]
Jul 06 20:20:42.655861 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:42.655826    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" path="/var/lib/kubelet/pods/2c995d26-042b-4c93-8d70-a59fe8f2b98a/volumes"
Jul 06 20:20:42.656410 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:20:42.656389    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" path="/var/lib/kubelet/pods/c00f50f3-8cf2-4232-ba49-a49af4c603f2/volumes"
Jul 06 20:21:09.419223 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.419190    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:21:09.419759 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.419403    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" podUID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" containerName="maas-api" containerID="cri-o://9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024" gracePeriod=30
Jul 06 20:21:09.665485 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.665462    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:21:09.818451 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.818424    2565 generic.go:356] "Generic (PLEG): container finished" podID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" containerID="9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024" exitCode=0
Jul 06 20:21:09.818611 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.818464    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"
Jul 06 20:21:09.818611 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.818472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" event={"ID":"11b5fa46-8723-4a17-94cb-86a2cf4aa1df","Type":"ContainerDied","Data":"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"}
Jul 06 20:21:09.818611 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.818538    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz" event={"ID":"11b5fa46-8723-4a17-94cb-86a2cf4aa1df","Type":"ContainerDied","Data":"964aaf74c53a9d2afe6c8573bef29c85f628096dd65ffb009c42bd8f563579ba"}
Jul 06 20:21:09.818611 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.818564    2565 scope.go:117] "RemoveContainer" containerID="9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"
Jul 06 20:21:09.821698 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.821673    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca\") pod \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") "
Jul 06 20:21:09.821807 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.821791    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") pod \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") "
Jul 06 20:21:09.821856 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.821833    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfkxb\" (UniqueName: \"kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb\") pod \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\" (UID: \"11b5fa46-8723-4a17-94cb-86a2cf4aa1df\") "
Jul 06 20:21:09.822057 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.822031    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11b5fa46-8723-4a17-94cb-86a2cf4aa1df" (UID: "11b5fa46-8723-4a17-94cb-86a2cf4aa1df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:21:09.824047 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.824018    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11b5fa46-8723-4a17-94cb-86a2cf4aa1df" (UID: "11b5fa46-8723-4a17-94cb-86a2cf4aa1df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:21:09.824152 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.824045    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb" (OuterVolumeSpecName: "kube-api-access-pfkxb") pod "11b5fa46-8723-4a17-94cb-86a2cf4aa1df" (UID: "11b5fa46-8723-4a17-94cb-86a2cf4aa1df"). InnerVolumeSpecName "kube-api-access-pfkxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:09.827700 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.827681    2565 scope.go:117] "RemoveContainer" containerID="9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"
Jul 06 20:21:09.828004 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:21:09.827977    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024\": container with ID starting with 9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024 not found: ID does not exist" containerID="9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"
Jul 06 20:21:09.828083 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.828015    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024"} err="failed to get container status \"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024\": rpc error: code = NotFound desc = could not find container \"9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024\": container with ID starting with 9e2f214eaec5d1a32804cf5c8baf7ae28adfb908b2fcbea20b2384c94d915024 not found: ID does not exist"
Jul 06 20:21:09.923130 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.923106    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:09.923130 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.923127    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:09.923274 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:09.923136    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfkxb\" (UniqueName: \"kubernetes.io/projected/11b5fa46-8723-4a17-94cb-86a2cf4aa1df-kube-api-access-pfkxb\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:10.141146 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:10.141117    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:21:10.146751 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:10.146729    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ec5878-65588d5cbc-cfzjz"]
Jul 06 20:21:10.656496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:10.656460    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" path="/var/lib/kubelet/pods/11b5fa46-8723-4a17-94cb-86a2cf4aa1df/volumes"
Jul 06 20:21:12.397481 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:12.397443    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:21:19.015086 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015009    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:19.015563 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015544    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" containerName="maas-api"
Jul 06 20:21:19.015608 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015567    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" containerName="maas-api"
Jul 06 20:21:19.015608 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015582    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" containerName="main"
Jul 06 20:21:19.015608 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015591    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" containerName="main"
Jul 06 20:21:19.015716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015616    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" containerName="main"
Jul 06 20:21:19.015716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015625    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" containerName="main"
Jul 06 20:21:19.015716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015641    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de484a37-e521-4db3-9890-c1161f97bf81" containerName="maas-api"
Jul 06 20:21:19.015716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015649    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de484a37-e521-4db3-9890-c1161f97bf81" containerName="maas-api"
Jul 06 20:21:19.015860 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015739    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00f50f3-8cf2-4232-ba49-a49af4c603f2" containerName="main"
Jul 06 20:21:19.015860 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015768    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b5fa46-8723-4a17-94cb-86a2cf4aa1df" containerName="maas-api"
Jul 06 20:21:19.015860 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015780    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c995d26-042b-4c93-8d70-a59fe8f2b98a" containerName="main"
Jul 06 20:21:19.015860 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.015789    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de484a37-e521-4db3-9890-c1161f97bf81" containerName="maas-api"
Jul 06 20:21:19.023026 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.022996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.026192 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.026171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-1a31be-serving-cert\""
Jul 06 20:21:19.026591 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.026566    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:21:19.031848 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.031824    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:19.201045 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.201006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.201217 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.201090    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.201217 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.201124    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncv5b\" (UniqueName: \"kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.302368 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.302279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.302368 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.302336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncv5b\" (UniqueName: \"kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.302550 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.302378    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.303084 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.303059    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.304716 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.304692    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.310899 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.310875    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncv5b\" (UniqueName: \"kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b\") pod \"maas-api-e2e-api-a-1a31be-c879987bb-lqgrs\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") " pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.335368 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.335344    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:19.470320 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.470291    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:19.471345 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:21:19.471320    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9062ef_2372_48f5_a266_ce38810ccf6b.slice/crio-46ae64f062bb73c0a91d9d495d249c8fa1fbb397af8555d077829896ecf36f4b WatchSource:0}: Error finding container 46ae64f062bb73c0a91d9d495d249c8fa1fbb397af8555d077829896ecf36f4b: Status 404 returned error can't find the container with id 46ae64f062bb73c0a91d9d495d249c8fa1fbb397af8555d077829896ecf36f4b
Jul 06 20:21:19.871849 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:19.871816    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" event={"ID":"9c9062ef-2372-48f5-a266-ce38810ccf6b","Type":"ContainerStarted","Data":"46ae64f062bb73c0a91d9d495d249c8fa1fbb397af8555d077829896ecf36f4b"}
Jul 06 20:21:20.878148 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:20.878112    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" event={"ID":"9c9062ef-2372-48f5-a266-ce38810ccf6b","Type":"ContainerStarted","Data":"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"}
Jul 06 20:21:20.878551 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:20.878190    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:20.896946 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:20.896886    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" podStartSLOduration=2.409571564 podStartE2EDuration="2.89687246s" totalImagesPullingTime="487.300896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:20.894752182 +0000 UTC m=+1979.156731045" watchObservedRunningTime="2026-07-06 20:21:20.89687246 +0000 UTC m=+1979.158851329"
Jul 06 20:21:26.888982 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:26.888953    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:27.074378 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.074345    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:27.077775 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.077751    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.080688 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.080664    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-6c248c-serving-cert\""
Jul 06 20:21:27.090493 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.090470    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:27.124207 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.124178    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:21:27.169828 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.169762    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.169828 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.169797    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.169828 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.169829    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qjj7\" (UniqueName: \"kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.271159 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.271127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.271325 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.271168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.271325 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.271188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qjj7\" (UniqueName: \"kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.271878 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.271855    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.273533 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.273513    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.282980 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.282959    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qjj7\" (UniqueName: \"kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7\") pod \"maas-api-e2e-api-b-6c248c-69855c89b6-nsppj\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") " pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.392067 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.392033    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:27.527798 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.527773    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:27.529851 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:21:27.529816    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9379aaa_502c_4081_9945_c9e8e315fd44.slice/crio-f8285a8736210d8db8b7aa520be262d8554cac62d3b05e906127ee8ff75494e0 WatchSource:0}: Error finding container f8285a8736210d8db8b7aa520be262d8554cac62d3b05e906127ee8ff75494e0: Status 404 returned error can't find the container with id f8285a8736210d8db8b7aa520be262d8554cac62d3b05e906127ee8ff75494e0
Jul 06 20:21:27.919437 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:27.919401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" event={"ID":"f9379aaa-502c-4081-9945-c9e8e315fd44","Type":"ContainerStarted","Data":"f8285a8736210d8db8b7aa520be262d8554cac62d3b05e906127ee8ff75494e0"}
Jul 06 20:21:28.926371 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:28.926334    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" event={"ID":"f9379aaa-502c-4081-9945-c9e8e315fd44","Type":"ContainerStarted","Data":"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"}
Jul 06 20:21:28.926754 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:28.926408    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:28.947506 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:28.947461    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" podStartSLOduration=1.491433526 podStartE2EDuration="1.947445243s" totalImagesPullingTime="456.011717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:28.946435063 +0000 UTC m=+1987.208413926" watchObservedRunningTime="2026-07-06 20:21:28.947445243 +0000 UTC m=+1987.209424123"
Jul 06 20:21:34.937093 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:34.937064    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:40.486067 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.486034    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:40.486500 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.486286    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" podUID="9c9062ef-2372-48f5-a266-ce38810ccf6b" containerName="maas-api" containerID="cri-o://4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f" gracePeriod=30
Jul 06 20:21:40.740028 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.739966    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:40.787083 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.787047    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncv5b\" (UniqueName: \"kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b\") pod \"9c9062ef-2372-48f5-a266-ce38810ccf6b\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") "
Jul 06 20:21:40.787264 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.787141    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca\") pod \"9c9062ef-2372-48f5-a266-ce38810ccf6b\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") "
Jul 06 20:21:40.787264 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.787210    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls\") pod \"9c9062ef-2372-48f5-a266-ce38810ccf6b\" (UID: \"9c9062ef-2372-48f5-a266-ce38810ccf6b\") "
Jul 06 20:21:40.787526 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.787503    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c9062ef-2372-48f5-a266-ce38810ccf6b" (UID: "9c9062ef-2372-48f5-a266-ce38810ccf6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:21:40.789145 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.789118    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c9062ef-2372-48f5-a266-ce38810ccf6b" (UID: "9c9062ef-2372-48f5-a266-ce38810ccf6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:21:40.789264 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.789239    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b" (OuterVolumeSpecName: "kube-api-access-ncv5b") pod "9c9062ef-2372-48f5-a266-ce38810ccf6b" (UID: "9c9062ef-2372-48f5-a266-ce38810ccf6b"). InnerVolumeSpecName "kube-api-access-ncv5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:40.888601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.888564    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncv5b\" (UniqueName: \"kubernetes.io/projected/9c9062ef-2372-48f5-a266-ce38810ccf6b-kube-api-access-ncv5b\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:40.888601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.888594    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c9062ef-2372-48f5-a266-ce38810ccf6b-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:40.888601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.888603    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c9062ef-2372-48f5-a266-ce38810ccf6b-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:40.992428 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.992338    2565 generic.go:356] "Generic (PLEG): container finished" podID="9c9062ef-2372-48f5-a266-ce38810ccf6b" containerID="4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f" exitCode=0
Jul 06 20:21:40.992428 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.992385    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"
Jul 06 20:21:40.992428 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.992420    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" event={"ID":"9c9062ef-2372-48f5-a266-ce38810ccf6b","Type":"ContainerDied","Data":"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"}
Jul 06 20:21:40.992650 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.992472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs" event={"ID":"9c9062ef-2372-48f5-a266-ce38810ccf6b","Type":"ContainerDied","Data":"46ae64f062bb73c0a91d9d495d249c8fa1fbb397af8555d077829896ecf36f4b"}
Jul 06 20:21:40.992650 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:40.992500    2565 scope.go:117] "RemoveContainer" containerID="4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"
Jul 06 20:21:41.001725 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:41.001707    2565 scope.go:117] "RemoveContainer" containerID="4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"
Jul 06 20:21:41.001995 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:21:41.001975    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f\": container with ID starting with 4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f not found: ID does not exist" containerID="4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"
Jul 06 20:21:41.002052 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:41.002003    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f"} err="failed to get container status \"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f\": rpc error: code = NotFound desc = could not find container \"4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f\": container with ID starting with 4d79e82ece2100d4491290b282423dee9c61c4eb97d3ecdeb35f42a08c70211f not found: ID does not exist"
Jul 06 20:21:41.013245 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:41.013218    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:41.017315 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:41.017290    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1a31be-c879987bb-lqgrs"]
Jul 06 20:21:42.663362 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:42.663331    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c9062ef-2372-48f5-a266-ce38810ccf6b" path="/var/lib/kubelet/pods/9c9062ef-2372-48f5-a266-ce38810ccf6b/volumes"
Jul 06 20:21:43.606136 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:43.606100    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:21:47.607027 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.606992    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:47.607411 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.607228    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" podUID="f9379aaa-502c-4081-9945-c9e8e315fd44" containerName="maas-api" containerID="cri-o://a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791" gracePeriod=30
Jul 06 20:21:47.848531 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.848506    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:47.944157 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.944086    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qjj7\" (UniqueName: \"kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7\") pod \"f9379aaa-502c-4081-9945-c9e8e315fd44\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") "
Jul 06 20:21:47.944157 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.944126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls\") pod \"f9379aaa-502c-4081-9945-c9e8e315fd44\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") "
Jul 06 20:21:47.944157 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.944153    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca\") pod \"f9379aaa-502c-4081-9945-c9e8e315fd44\" (UID: \"f9379aaa-502c-4081-9945-c9e8e315fd44\") "
Jul 06 20:21:47.944535 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.944506    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9379aaa-502c-4081-9945-c9e8e315fd44" (UID: "f9379aaa-502c-4081-9945-c9e8e315fd44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:21:47.946119 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.946099    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9379aaa-502c-4081-9945-c9e8e315fd44" (UID: "f9379aaa-502c-4081-9945-c9e8e315fd44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:21:47.946223 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:47.946113    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7" (OuterVolumeSpecName: "kube-api-access-7qjj7") pod "f9379aaa-502c-4081-9945-c9e8e315fd44" (UID: "f9379aaa-502c-4081-9945-c9e8e315fd44"). InnerVolumeSpecName "kube-api-access-7qjj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:48.036851 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.036824    2565 generic.go:356] "Generic (PLEG): container finished" podID="f9379aaa-502c-4081-9945-c9e8e315fd44" containerID="a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791" exitCode=0
Jul 06 20:21:48.037006 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.036866    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"
Jul 06 20:21:48.037006 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.036902    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" event={"ID":"f9379aaa-502c-4081-9945-c9e8e315fd44","Type":"ContainerDied","Data":"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"}
Jul 06 20:21:48.037006 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.036960    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj" event={"ID":"f9379aaa-502c-4081-9945-c9e8e315fd44","Type":"ContainerDied","Data":"f8285a8736210d8db8b7aa520be262d8554cac62d3b05e906127ee8ff75494e0"}
Jul 06 20:21:48.037006 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.036982    2565 scope.go:117] "RemoveContainer" containerID="a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"
Jul 06 20:21:48.044692 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.044669    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qjj7\" (UniqueName: \"kubernetes.io/projected/f9379aaa-502c-4081-9945-c9e8e315fd44-kube-api-access-7qjj7\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:48.044802 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.044696    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9379aaa-502c-4081-9945-c9e8e315fd44-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:48.044802 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.044710    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9379aaa-502c-4081-9945-c9e8e315fd44-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:21:48.046250 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.046228    2565 scope.go:117] "RemoveContainer" containerID="a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"
Jul 06 20:21:48.046459 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:21:48.046440    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791\": container with ID starting with a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791 not found: ID does not exist" containerID="a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"
Jul 06 20:21:48.046503 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.046467    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791"} err="failed to get container status \"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791\": rpc error: code = NotFound desc = could not find container \"a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791\": container with ID starting with a6f78323a4b419f65135b9c8caf70f552c1c463ac7de99acd24bbcbdff1cd791 not found: ID does not exist"
Jul 06 20:21:48.058228 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.058208    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:48.062679 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.062659    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c248c-69855c89b6-nsppj"]
Jul 06 20:21:48.655366 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:48.655333    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9379aaa-502c-4081-9945-c9e8e315fd44" path="/var/lib/kubelet/pods/f9379aaa-502c-4081-9945-c9e8e315fd44/volumes"
Jul 06 20:21:56.994103 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:56.994070    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:21:57.237428 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.237391    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:21:57.237921 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.237895    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9379aaa-502c-4081-9945-c9e8e315fd44" containerName="maas-api"
Jul 06 20:21:57.237976 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.237927    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9379aaa-502c-4081-9945-c9e8e315fd44" containerName="maas-api"
Jul 06 20:21:57.237976 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.237947    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c9062ef-2372-48f5-a266-ce38810ccf6b" containerName="maas-api"
Jul 06 20:21:57.237976 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.237956    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c9062ef-2372-48f5-a266-ce38810ccf6b" containerName="maas-api"
Jul 06 20:21:57.238069 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.238049    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c9062ef-2372-48f5-a266-ce38810ccf6b" containerName="maas-api"
Jul 06 20:21:57.238069 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.238064    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9379aaa-502c-4081-9945-c9e8e315fd44" containerName="maas-api"
Jul 06 20:21:57.241501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.241483    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.243989 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.243964    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-c46beb-serving-cert\""
Jul 06 20:21:57.244141 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.244034    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\""
Jul 06 20:21:57.250426 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.250388    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:21:57.323354 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.323325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q45hr\" (UniqueName: \"kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.323469 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.323378    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.323469 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.323402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.424052 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.424024    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.424169 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.424058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.424169 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.424157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q45hr\" (UniqueName: \"kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.424293 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:21:57.424240    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-c46beb-serving-cert: secret "maas-api-e2e-shared-a-c46beb-serving-cert" not found
Jul 06 20:21:57.424345 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:21:57.424326    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls podName:2a4743ce-8514-4ad0-9123-edc7129d0d84 nodeName:}" failed. No retries permitted until 2026-07-06 20:21:57.924304809 +0000 UTC m=+2016.186283662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls") pod "maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" (UID: "2a4743ce-8514-4ad0-9123-edc7129d0d84") : secret "maas-api-e2e-shared-a-c46beb-serving-cert" not found
Jul 06 20:21:57.424694 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.424676    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.435228 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.435206    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q45hr\" (UniqueName: \"kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.927209 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.927175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:57.929496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:57.929479    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") pod \"maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") " pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:58.154116 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:58.154086    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:58.285962 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:58.285938    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:21:58.287703 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:21:58.287672    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4743ce_8514_4ad0_9123_edc7129d0d84.slice/crio-2f8fd70672c86a7cc78c82d647b0d0565eabadc0b8b290e55e13fb9e9de410d2 WatchSource:0}: Error finding container 2f8fd70672c86a7cc78c82d647b0d0565eabadc0b8b290e55e13fb9e9de410d2: Status 404 returned error can't find the container with id 2f8fd70672c86a7cc78c82d647b0d0565eabadc0b8b290e55e13fb9e9de410d2
Jul 06 20:21:59.097369 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:59.097333    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" event={"ID":"2a4743ce-8514-4ad0-9123-edc7129d0d84","Type":"ContainerStarted","Data":"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"}
Jul 06 20:21:59.097552 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:59.097388    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" event={"ID":"2a4743ce-8514-4ad0-9123-edc7129d0d84","Type":"ContainerStarted","Data":"2f8fd70672c86a7cc78c82d647b0d0565eabadc0b8b290e55e13fb9e9de410d2"}
Jul 06 20:21:59.097552 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:59.097416    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:21:59.115692 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:21:59.115641    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" podStartSLOduration=1.728484819 podStartE2EDuration="2.115627107s" totalImagesPullingTime="387.142288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:59.114631752 +0000 UTC m=+2017.376610618" watchObservedRunningTime="2026-07-06 20:21:59.115627107 +0000 UTC m=+2017.377605970"
Jul 06 20:22:05.107191 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.107161    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:22:05.413366 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.413293    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:22:05.416865 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.416849    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.419172 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.419151    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-a2776d-serving-cert\""
Jul 06 20:22:05.427327 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.427307    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:22:05.592624 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.592588    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.592769 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.592651    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.592769 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.592686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64jb9\" (UniqueName: \"kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.693285 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.693222    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64jb9\" (UniqueName: \"kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.693285 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.693275    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.693421 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.693313    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.693456 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:22:05.693427    2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-a2776d-serving-cert: secret "maas-api-e2e-shared-b-a2776d-serving-cert" not found
Jul 06 20:22:05.693512 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:22:05.693499    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls podName:d2c245c5-9300-4719-ab9d-d392d59dadef nodeName:}" failed. No retries permitted until 2026-07-06 20:22:06.19347126 +0000 UTC m=+2024.455450112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls") pod "maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" (UID: "d2c245c5-9300-4719-ab9d-d392d59dadef") : secret "maas-api-e2e-shared-b-a2776d-serving-cert" not found
Jul 06 20:22:05.693896 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.693878    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:05.722039 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:05.722018    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64jb9\" (UniqueName: \"kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:06.198606 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:06.198576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:06.200952 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:06.200928    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") pod \"maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") " pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:06.327231 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:06.327202    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:06.457081 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:06.457011    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:22:06.460098 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:22:06.460068    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c245c5_9300_4719_ab9d_d392d59dadef.slice/crio-e455bdc5e03930a44253313d82a95bf04ea2d26befef92e5cfe861a22db67d00 WatchSource:0}: Error finding container e455bdc5e03930a44253313d82a95bf04ea2d26befef92e5cfe861a22db67d00: Status 404 returned error can't find the container with id e455bdc5e03930a44253313d82a95bf04ea2d26befef92e5cfe861a22db67d00
Jul 06 20:22:07.140439 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:07.140396    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" event={"ID":"d2c245c5-9300-4719-ab9d-d392d59dadef","Type":"ContainerStarted","Data":"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"}
Jul 06 20:22:07.140628 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:07.140488    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" event={"ID":"d2c245c5-9300-4719-ab9d-d392d59dadef","Type":"ContainerStarted","Data":"e455bdc5e03930a44253313d82a95bf04ea2d26befef92e5cfe861a22db67d00"}
Jul 06 20:22:07.140628 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:07.140518    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:07.159294 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:07.159209    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" podStartSLOduration=1.723333531 podStartE2EDuration="2.15919504s" totalImagesPullingTime="435.861509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:07.157254603 +0000 UTC m=+2025.419233479" watchObservedRunningTime="2026-07-06 20:22:07.15919504 +0000 UTC m=+2025.421173948"
Jul 06 20:22:09.692055 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:09.692021    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:22:13.152265 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:13.152228    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:22:18.285260 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.285222    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:22:18.288897 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.288879    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.291186 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.291166    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c46beb\"/\"openshift-service-ca.crt\""
Jul 06 20:22:18.292210 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.292189    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c46beb\"/\"default-dockercfg-vfhx2\""
Jul 06 20:22:18.292276 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.292197    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c46beb\"/\"kube-root-ca.crt\""
Jul 06 20:22:18.292276 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.292238    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c46beb\"/\"auth-test-model-c46beb-kserve-self-signed-certs\""
Jul 06 20:22:18.300334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.300311    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:22:18.390601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.390753 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.390753 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.390753 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.390949 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9x9s\" (UniqueName: \"kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.390949 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.390849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491434 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491568 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491443    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491568 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9x9s\" (UniqueName: \"kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491568 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491568 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491568 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491879 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491857    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491978 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491888    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.491978 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.491960    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.493674 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.493650    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.494047 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.494029    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.498773 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.498751    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9x9s\" (UniqueName: \"kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s\") pod \"auth-test-model-c46beb-kserve-6f8d74c9f-9lngl\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") " pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.597346 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.597280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:18.733948 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:18.733903    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:22:18.734884 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:22:18.734849    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51a62fe9_6f37_4ab7_ab77_397d88d8fab5.slice/crio-0e3b3dfd59aefd7f0edbb62e49a4c18511427bc270adb45f16b3d41b9409b179 WatchSource:0}: Error finding container 0e3b3dfd59aefd7f0edbb62e49a4c18511427bc270adb45f16b3d41b9409b179: Status 404 returned error can't find the container with id 0e3b3dfd59aefd7f0edbb62e49a4c18511427bc270adb45f16b3d41b9409b179
Jul 06 20:22:19.205684 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:19.205646    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" event={"ID":"51a62fe9-6f37-4ab7-ab77-397d88d8fab5","Type":"ContainerStarted","Data":"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"}
Jul 06 20:22:19.205859 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:19.205692    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" event={"ID":"51a62fe9-6f37-4ab7-ab77-397d88d8fab5","Type":"ContainerStarted","Data":"0e3b3dfd59aefd7f0edbb62e49a4c18511427bc270adb45f16b3d41b9409b179"}
Jul 06 20:22:19.223610 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:19.223522    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" podStartSLOduration=1.223508531 podStartE2EDuration="1.223508531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:19.222976498 +0000 UTC m=+2037.484955362" watchObservedRunningTime="2026-07-06 20:22:19.223508531 +0000 UTC m=+2037.485487398"
Jul 06 20:22:23.207587 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:23.207551    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:22:28.597986 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:28.597937    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:28.598388 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:28.598004    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:28.610135 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:28.610109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:29.268714 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:29.268683    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:22:42.703236 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.703202    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:22:42.707253 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.707231    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.709636 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.709605    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"kube-root-ca.crt\""
Jul 06 20:22:42.709636 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.709625    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"openshift-service-ca.crt\""
Jul 06 20:22:42.709827 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.709741    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"default-dockercfg-m642m\""
Jul 06 20:22:42.710897 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.710872    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"auth-test-model-a2776d-kserve-self-signed-certs\""
Jul 06 20:22:42.716575 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.716554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:22:42.798789 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.798764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.798953 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.798828    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.799035 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.798949    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.799035 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.798988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.799145 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.799034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2pd\" (UniqueName: \"kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.799145 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.799115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899617 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899717 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899642    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899717 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899687    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899718    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2pd\" (UniqueName: \"kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.899819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.899787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.900150 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.900122    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.900243 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.900153    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.900243 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.900153    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.901839 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.901820    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.902110 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.902093    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:42.907715 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:42.907693    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2pd\" (UniqueName: \"kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd\") pod \"auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") " pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:43.017110 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:43.017084    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:43.148220 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:43.148197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:22:43.150384 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:22:43.150356    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29850ac7_c8a2_4393_b1fe_59b045063bd9.slice/crio-7f9d0279c1d77d24b69805a1697844ab3eac0801585de91e4f506db2eecb07a6 WatchSource:0}: Error finding container 7f9d0279c1d77d24b69805a1697844ab3eac0801585de91e4f506db2eecb07a6: Status 404 returned error can't find the container with id 7f9d0279c1d77d24b69805a1697844ab3eac0801585de91e4f506db2eecb07a6
Jul 06 20:22:43.336512 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:43.336417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" event={"ID":"29850ac7-c8a2-4393-b1fe-59b045063bd9","Type":"ContainerStarted","Data":"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"}
Jul 06 20:22:43.336512 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:43.336461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" event={"ID":"29850ac7-c8a2-4393-b1fe-59b045063bd9","Type":"ContainerStarted","Data":"7f9d0279c1d77d24b69805a1697844ab3eac0801585de91e4f506db2eecb07a6"}
Jul 06 20:22:43.355982 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:43.355933    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" podStartSLOduration=1.355919633 podStartE2EDuration="1.355919633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:43.353373321 +0000 UTC m=+2061.615352186" watchObservedRunningTime="2026-07-06 20:22:43.355919633 +0000 UTC m=+2061.617898487"
Jul 06 20:22:46.204845 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:46.204768    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:22:53.018065 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:53.018031    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:53.018462 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:53.018324    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:53.029709 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:53.029690    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:22:53.399459 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:22:53.399387    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:23:05.711463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:05.711428    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:23:22.771950 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:22.771900    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:23:22.779313 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:22.779294    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:23:23.203315 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:23.203238    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:23:36.796946 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:36.796884    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:23:54.105387 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:23:54.105351    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:24:07.797958 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:07.797902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:24:22.602778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:22.602699    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:24:36.067702 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:36.067658    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:24:36.976664 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:36.976624    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:24:39.302083 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:39.302052    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:24:47.311896 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.311860    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:24:47.317444 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.317416    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.319985 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.319961    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c46beb\"/\"sub-test-model-c46beb-kserve-self-signed-certs\""
Jul 06 20:24:47.327616 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.327592    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:24:47.432708 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.432819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.432819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.432819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432786    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8l78\" (UniqueName: \"kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.432949 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.432949 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.432904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534218 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534332 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534332 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8l78\" (UniqueName: \"kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534332 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534290    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534332 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534305    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534541 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534603 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534577    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534696 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534678    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.534789 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.534772    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.536504 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.536481    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.536736 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.536718    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.541733 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.541717    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8l78\" (UniqueName: \"kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78\") pod \"sub-test-model-c46beb-kserve-6fbc4f6775-65mb4\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") " pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.627202 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.627143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:47.762495 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:47.762466    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:24:47.764127 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:24:47.764101    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b9ad333_f936_4eac_9299_6401258c7acc.slice/crio-3daebb4cbb694ec65bda2776ee698204327317ca1461d234d1a5e7a7a0b4711c WatchSource:0}: Error finding container 3daebb4cbb694ec65bda2776ee698204327317ca1461d234d1a5e7a7a0b4711c: Status 404 returned error can't find the container with id 3daebb4cbb694ec65bda2776ee698204327317ca1461d234d1a5e7a7a0b4711c
Jul 06 20:24:48.003130 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:48.003040    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" event={"ID":"7b9ad333-f936-4eac-9299-6401258c7acc","Type":"ContainerStarted","Data":"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"}
Jul 06 20:24:48.003130 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:48.003082    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" event={"ID":"7b9ad333-f936-4eac-9299-6401258c7acc","Type":"ContainerStarted","Data":"3daebb4cbb694ec65bda2776ee698204327317ca1461d234d1a5e7a7a0b4711c"}
Jul 06 20:24:48.020141 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:48.020092    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" podStartSLOduration=1.02007691 podStartE2EDuration="1.02007691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:48.019351986 +0000 UTC m=+2186.281330861" watchObservedRunningTime="2026-07-06 20:24:48.02007691 +0000 UTC m=+2186.282055774"
Jul 06 20:24:51.086829 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.086769    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" podUID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" containerName="main" containerID="cri-o://f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1" gracePeriod=45
Jul 06 20:24:51.337534 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.337475    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:24:51.469360 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469334    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469360 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469364    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469388    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9x9s\" (UniqueName: \"kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469418    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469442    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469601 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469469    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs\") pod \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\" (UID: \"51a62fe9-6f37-4ab7-ab77-397d88d8fab5\") "
Jul 06 20:24:51.469778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469673    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache" (OuterVolumeSpecName: "model-cache") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:51.469778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469657    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home" (OuterVolumeSpecName: "home") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:51.469778 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.469736    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:51.471501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.471480    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s" (OuterVolumeSpecName: "kube-api-access-s9x9s") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "kube-api-access-s9x9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:24:51.471591 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.471497    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:24:51.471591 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.471554    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm" (OuterVolumeSpecName: "dshm") pod "51a62fe9-6f37-4ab7-ab77-397d88d8fab5" (UID: "51a62fe9-6f37-4ab7-ab77-397d88d8fab5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:51.570243 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570210    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.570243 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570241    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.570388 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570255    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9x9s\" (UniqueName: \"kubernetes.io/projected/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-kube-api-access-s9x9s\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.570388 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570272    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.570388 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570280    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.570388 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.570289    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a62fe9-6f37-4ab7-ab77-397d88d8fab5-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:51.996253 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:51.996217    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" podUID="29850ac7-c8a2-4393-b1fe-59b045063bd9" containerName="main" containerID="cri-o://5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99" gracePeriod=45
Jul 06 20:24:52.024678 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.024649    2565 generic.go:356] "Generic (PLEG): container finished" podID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" containerID="f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1" exitCode=0
Jul 06 20:24:52.024794 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.024708    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"
Jul 06 20:24:52.024794 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.024738    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" event={"ID":"51a62fe9-6f37-4ab7-ab77-397d88d8fab5","Type":"ContainerDied","Data":"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"}
Jul 06 20:24:52.024794 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.024792    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl" event={"ID":"51a62fe9-6f37-4ab7-ab77-397d88d8fab5","Type":"ContainerDied","Data":"0e3b3dfd59aefd7f0edbb62e49a4c18511427bc270adb45f16b3d41b9409b179"}
Jul 06 20:24:52.024984 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.024816    2565 scope.go:117] "RemoveContainer" containerID="f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"
Jul 06 20:24:52.062448 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.062427    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:24:52.067182 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.067161    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/auth-test-model-c46beb-kserve-6f8d74c9f-9lngl"]
Jul 06 20:24:52.109579 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.109386    2565 scope.go:117] "RemoveContainer" containerID="f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"
Jul 06 20:24:52.109862 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:24:52.109842    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1\": container with ID starting with f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1 not found: ID does not exist" containerID="f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"
Jul 06 20:24:52.109936 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.109872    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1"} err="failed to get container status \"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1\": rpc error: code = NotFound desc = could not find container \"f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1\": container with ID starting with f70153e9df031e45d666e1b22f6ed5a46661a3c3287901236b56a4e0d2318dd1 not found: ID does not exist"
Jul 06 20:24:52.236509 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.236487    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:24:52.377096 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377229 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377105    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg2pd\" (UniqueName: \"kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377229 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377132    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377229 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377150    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377229 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377203    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377447 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377242    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs\") pod \"29850ac7-c8a2-4393-b1fe-59b045063bd9\" (UID: \"29850ac7-c8a2-4393-b1fe-59b045063bd9\") "
Jul 06 20:24:52.377447 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377394    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache" (OuterVolumeSpecName: "model-cache") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:52.377447 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377435    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home" (OuterVolumeSpecName: "home") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:52.377593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377507    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:52.377593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377570    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.377593 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377588    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.377704 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.377600    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.379402 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.379380    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:24:52.379570 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.379552    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm" (OuterVolumeSpecName: "dshm") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:24:52.379631 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.379553    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd" (OuterVolumeSpecName: "kube-api-access-qg2pd") pod "29850ac7-c8a2-4393-b1fe-59b045063bd9" (UID: "29850ac7-c8a2-4393-b1fe-59b045063bd9"). InnerVolumeSpecName "kube-api-access-qg2pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:24:52.478721 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.478699    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29850ac7-c8a2-4393-b1fe-59b045063bd9-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.478721 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.478719    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg2pd\" (UniqueName: \"kubernetes.io/projected/29850ac7-c8a2-4393-b1fe-59b045063bd9-kube-api-access-qg2pd\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.478858 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.478729    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29850ac7-c8a2-4393-b1fe-59b045063bd9-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:24:52.655661 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:52.655587    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" path="/var/lib/kubelet/pods/51a62fe9-6f37-4ab7-ab77-397d88d8fab5/volumes"
Jul 06 20:24:53.031819 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.031789    2565 generic.go:356] "Generic (PLEG): container finished" podID="29850ac7-c8a2-4393-b1fe-59b045063bd9" containerID="5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99" exitCode=0
Jul 06 20:24:53.032008 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.031833    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"
Jul 06 20:24:53.032008 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.031877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" event={"ID":"29850ac7-c8a2-4393-b1fe-59b045063bd9","Type":"ContainerDied","Data":"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"}
Jul 06 20:24:53.032008 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.031933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg" event={"ID":"29850ac7-c8a2-4393-b1fe-59b045063bd9","Type":"ContainerDied","Data":"7f9d0279c1d77d24b69805a1697844ab3eac0801585de91e4f506db2eecb07a6"}
Jul 06 20:24:53.032008 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.031953    2565 scope.go:117] "RemoveContainer" containerID="5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"
Jul 06 20:24:53.040615 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.040599    2565 scope.go:117] "RemoveContainer" containerID="5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"
Jul 06 20:24:53.040847 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:24:53.040830    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99\": container with ID starting with 5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99 not found: ID does not exist" containerID="5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"
Jul 06 20:24:53.040889 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.040857    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99"} err="failed to get container status \"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99\": rpc error: code = NotFound desc = could not find container \"5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99\": container with ID starting with 5bfdde9f0ba77fdbb3b2d1b8e3087160c9f72b1d2135164864556b46e599cb99 not found: ID does not exist"
Jul 06 20:24:53.048262 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.048241    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:24:53.052416 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.052397    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/auth-test-model-a2776d-kserve-68c5c9c5c9-sh8bg"]
Jul 06 20:24:53.101510 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:53.101486    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:24:54.655502 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:54.655472    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29850ac7-c8a2-4393-b1fe-59b045063bd9" path="/var/lib/kubelet/pods/29850ac7-c8a2-4393-b1fe-59b045063bd9/volumes"
Jul 06 20:24:57.628245 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:57.628213    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:57.628684 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:57.628261    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:57.640006 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:57.639982    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:24:58.069186 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:24:58.069153    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:25:11.202441 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202409    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:25:11.202808 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202763    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29850ac7-c8a2-4393-b1fe-59b045063bd9" containerName="main"
Jul 06 20:25:11.202808 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202774    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="29850ac7-c8a2-4393-b1fe-59b045063bd9" containerName="main"
Jul 06 20:25:11.202808 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202783    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" containerName="main"
Jul 06 20:25:11.202808 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202787    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" containerName="main"
Jul 06 20:25:11.203074 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202847    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a62fe9-6f37-4ab7-ab77-397d88d8fab5" containerName="main"
Jul 06 20:25:11.203074 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.202856    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="29850ac7-c8a2-4393-b1fe-59b045063bd9" containerName="main"
Jul 06 20:25:11.205889 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.205871    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.209938 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.209881    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"default-dockercfg-m642m\""
Jul 06 20:25:11.209938 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.209893    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"sub-test-model-a2776d-kserve-self-signed-certs\""
Jul 06 20:25:11.210136 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.209981    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"openshift-service-ca.crt\""
Jul 06 20:25:11.210136 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.209981    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"kube-root-ca.crt\""
Jul 06 20:25:11.216208 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.216188    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:25:11.342883 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.342853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.343030 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.342898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.343030 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.342972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.343030 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.342991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.343139 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.343047    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.343139 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.343084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrghj\" (UniqueName: \"kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.443775 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.443875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.443875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443814    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.443875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.443875 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.444039 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.443902    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrghj\" (UniqueName: \"kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.444295 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.444269    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.444403 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.444319    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.444403 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.444342    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.446064 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.446037    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.446177 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.446159    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.453359 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.453305    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrghj\" (UniqueName: \"kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj\") pod \"sub-test-model-a2776d-kserve-5895989c97-9w64x\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") " pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.515173 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.515151    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:11.648504 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:11.648480    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:25:11.651266 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:25:11.651240    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65aa7ee9_36f2_4646_8de6_d87bb798457a.slice/crio-5fce3efb2ff3039608d2260086b3b8aac2e6a3bc7d543a90145cde48f37abc55 WatchSource:0}: Error finding container 5fce3efb2ff3039608d2260086b3b8aac2e6a3bc7d543a90145cde48f37abc55: Status 404 returned error can't find the container with id 5fce3efb2ff3039608d2260086b3b8aac2e6a3bc7d543a90145cde48f37abc55
Jul 06 20:25:12.131221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:12.131181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" event={"ID":"65aa7ee9-36f2-4646-8de6-d87bb798457a","Type":"ContainerStarted","Data":"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"}
Jul 06 20:25:12.131221 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:12.131233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" event={"ID":"65aa7ee9-36f2-4646-8de6-d87bb798457a","Type":"ContainerStarted","Data":"5fce3efb2ff3039608d2260086b3b8aac2e6a3bc7d543a90145cde48f37abc55"}
Jul 06 20:25:12.150996 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:12.150866    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" podStartSLOduration=1.15084915 podStartE2EDuration="1.15084915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:25:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:12.150326522 +0000 UTC m=+2210.412305384" watchObservedRunningTime="2026-07-06 20:25:12.15084915 +0000 UTC m=+2210.412828014"
Jul 06 20:25:14.804334 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:14.804302    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:25:21.515365 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:21.515329    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:21.515835 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:21.515376    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:21.526950 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:21.526929    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:22.195517 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:22.195490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:25:29.401314 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:29.401277    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:25:45.469972 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:45.469874    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:25:46.422575 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:46.422488    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:25:49.599114 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:25:49.599082    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:26:00.489530 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.489488    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" podUID="7b9ad333-f936-4eac-9299-6401258c7acc" containerName="main" containerID="cri-o://5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b" gracePeriod=45
Jul 06 20:26:00.720520 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.720495    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:26:00.867028 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867000    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867035    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867053    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867095    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8l78\" (UniqueName: \"kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867134    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867203 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867167    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm\") pod \"7b9ad333-f936-4eac-9299-6401258c7acc\" (UID: \"7b9ad333-f936-4eac-9299-6401258c7acc\") "
Jul 06 20:26:00.867463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867323    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home" (OuterVolumeSpecName: "home") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:00.867463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867364    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:00.867463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867409    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:00.867463 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.867401    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:00.869176 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.869146    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm" (OuterVolumeSpecName: "dshm") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:00.869274 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.869182    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:26:00.869274 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.869217    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78" (OuterVolumeSpecName: "kube-api-access-b8l78") pod "7b9ad333-f936-4eac-9299-6401258c7acc" (UID: "7b9ad333-f936-4eac-9299-6401258c7acc"). InnerVolumeSpecName "kube-api-access-b8l78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:00.968350 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.968319    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9ad333-f936-4eac-9299-6401258c7acc-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:00.968350 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.968342    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:00.968350 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.968352    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8l78\" (UniqueName: \"kubernetes.io/projected/7b9ad333-f936-4eac-9299-6401258c7acc-kube-api-access-b8l78\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:00.968564 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.968361    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:00.968564 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:00.968370    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9ad333-f936-4eac-9299-6401258c7acc-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.390107 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.390076    2565 generic.go:356] "Generic (PLEG): container finished" podID="7b9ad333-f936-4eac-9299-6401258c7acc" containerID="5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b" exitCode=0
Jul 06 20:26:01.390256 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.390118    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"
Jul 06 20:26:01.390256 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.390129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" event={"ID":"7b9ad333-f936-4eac-9299-6401258c7acc","Type":"ContainerDied","Data":"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"}
Jul 06 20:26:01.390256 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.390160    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4" event={"ID":"7b9ad333-f936-4eac-9299-6401258c7acc","Type":"ContainerDied","Data":"3daebb4cbb694ec65bda2776ee698204327317ca1461d234d1a5e7a7a0b4711c"}
Jul 06 20:26:01.390256 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.390177    2565 scope.go:117] "RemoveContainer" containerID="5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"
Jul 06 20:26:01.399527 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.399510    2565 scope.go:117] "RemoveContainer" containerID="5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"
Jul 06 20:26:01.399783 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:26:01.399764    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b\": container with ID starting with 5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b not found: ID does not exist" containerID="5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"
Jul 06 20:26:01.399864 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.399790    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b"} err="failed to get container status \"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b\": rpc error: code = NotFound desc = could not find container \"5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b\": container with ID starting with 5bac9de3d84715c8b07e17f7ffb31c3c3cbb5c409d3a56c77c9ca2028d517b1b not found: ID does not exist"
Jul 06 20:26:01.415820 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.415796    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:26:01.419410 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.419385    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c46beb/sub-test-model-c46beb-kserve-6fbc4f6775-65mb4"]
Jul 06 20:26:01.446765 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.446720    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" podUID="65aa7ee9-36f2-4646-8de6-d87bb798457a" containerName="main" containerID="cri-o://40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066" gracePeriod=45
Jul 06 20:26:01.701306 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.701282    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:26:01.775157 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775130    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775280 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775280 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775204    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775280 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775244    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775397 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775302    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775397 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775331    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrghj\" (UniqueName: \"kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj\") pod \"65aa7ee9-36f2-4646-8de6-d87bb798457a\" (UID: \"65aa7ee9-36f2-4646-8de6-d87bb798457a\") "
Jul 06 20:26:01.775497 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775422    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home" (OuterVolumeSpecName: "home") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:01.775497 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775430    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache" (OuterVolumeSpecName: "model-cache") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:01.775602 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775519    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:01.775659 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775631    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.775659 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775645    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.775659 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.775652    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.777262 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.777240    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm" (OuterVolumeSpecName: "dshm") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:01.777362 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.777327    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj" (OuterVolumeSpecName: "kube-api-access-vrghj") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "kube-api-access-vrghj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:01.777362 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.777335    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65aa7ee9-36f2-4646-8de6-d87bb798457a" (UID: "65aa7ee9-36f2-4646-8de6-d87bb798457a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:26:01.876541 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.876521    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65aa7ee9-36f2-4646-8de6-d87bb798457a-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.876541 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.876542    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa7ee9-36f2-4646-8de6-d87bb798457a-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:01.876689 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:01.876552    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrghj\" (UniqueName: \"kubernetes.io/projected/65aa7ee9-36f2-4646-8de6-d87bb798457a-kube-api-access-vrghj\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:26:02.397127 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.397091    2565 generic.go:356] "Generic (PLEG): container finished" podID="65aa7ee9-36f2-4646-8de6-d87bb798457a" containerID="40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066" exitCode=0
Jul 06 20:26:02.397290 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.397135    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"
Jul 06 20:26:02.397290 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.397153    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" event={"ID":"65aa7ee9-36f2-4646-8de6-d87bb798457a","Type":"ContainerDied","Data":"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"}
Jul 06 20:26:02.397290 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.397190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x" event={"ID":"65aa7ee9-36f2-4646-8de6-d87bb798457a","Type":"ContainerDied","Data":"5fce3efb2ff3039608d2260086b3b8aac2e6a3bc7d543a90145cde48f37abc55"}
Jul 06 20:26:02.397290 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.397213    2565 scope.go:117] "RemoveContainer" containerID="40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"
Jul 06 20:26:02.406162 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.406142    2565 scope.go:117] "RemoveContainer" containerID="40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"
Jul 06 20:26:02.406439 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:26:02.406421    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066\": container with ID starting with 40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066 not found: ID does not exist" containerID="40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"
Jul 06 20:26:02.406488 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.406447    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066"} err="failed to get container status \"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066\": rpc error: code = NotFound desc = could not find container \"40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066\": container with ID starting with 40a8cb1ab34e6694e033dbe62f229df3a5f8d6d1b0c7d5f2394b2a191eabb066 not found: ID does not exist"
Jul 06 20:26:02.418975 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.418949    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:26:02.422040 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.422015    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/sub-test-model-a2776d-kserve-5895989c97-9w64x"]
Jul 06 20:26:02.655689 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.655611    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65aa7ee9-36f2-4646-8de6-d87bb798457a" path="/var/lib/kubelet/pods/65aa7ee9-36f2-4646-8de6-d87bb798457a/volumes"
Jul 06 20:26:02.656204 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:02.656186    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b9ad333-f936-4eac-9299-6401258c7acc" path="/var/lib/kubelet/pods/7b9ad333-f936-4eac-9299-6401258c7acc/volumes"
Jul 06 20:26:04.497025 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:04.496991    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:26:16.604769 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.604735    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:26:16.605282 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605261    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65aa7ee9-36f2-4646-8de6-d87bb798457a" containerName="main"
Jul 06 20:26:16.605361 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605286    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="65aa7ee9-36f2-4646-8de6-d87bb798457a" containerName="main"
Jul 06 20:26:16.605361 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605322    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9ad333-f936-4eac-9299-6401258c7acc" containerName="main"
Jul 06 20:26:16.605361 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605332    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9ad333-f936-4eac-9299-6401258c7acc" containerName="main"
Jul 06 20:26:16.605508 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605434    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9ad333-f936-4eac-9299-6401258c7acc" containerName="main"
Jul 06 20:26:16.605508 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.605452    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="65aa7ee9-36f2-4646-8de6-d87bb798457a" containerName="main"
Jul 06 20:26:16.609779 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.609757    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.612347 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.612326    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"kube-root-ca.crt\""
Jul 06 20:26:16.613387 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.613366    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"openshift-service-ca.crt\""
Jul 06 20:26:16.613503 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.613448    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"default-dockercfg-m642m\""
Jul 06 20:26:16.613503 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.613364    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2776d\"/\"rate-test-model-a2776d-kserve-self-signed-certs\""
Jul 06 20:26:16.630081 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.630051    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:26:16.804640 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.804788 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.804788 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4thcp\" (UniqueName: \"kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.804867 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804855    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.804926 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.805064 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.804983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905490 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905490 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4thcp\" (UniqueName: \"kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905676 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905676 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905526    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905676 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905676 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905896 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905873    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.905983 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905934    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.906022 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.905996    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.907799 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.907768    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.908000 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.907984    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.913192 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.913174    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4thcp\" (UniqueName: \"kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp\") pod \"rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") " pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:16.919036 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:16.919019    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:17.058496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:17.058473    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:26:17.060527 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:26:17.060501    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f481232_dca4_4e40_b956_f1739b1c911c.slice/crio-b89168b93be8c401577040c979fcb130c6a491354fb850045203acad176bf82a WatchSource:0}: Error finding container b89168b93be8c401577040c979fcb130c6a491354fb850045203acad176bf82a: Status 404 returned error can't find the container with id b89168b93be8c401577040c979fcb130c6a491354fb850045203acad176bf82a
Jul 06 20:26:17.475230 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:17.475196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" event={"ID":"1f481232-dca4-4e40-b956-f1739b1c911c","Type":"ContainerStarted","Data":"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"}
Jul 06 20:26:17.475415 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:17.475239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" event={"ID":"1f481232-dca4-4e40-b956-f1739b1c911c","Type":"ContainerStarted","Data":"b89168b93be8c401577040c979fcb130c6a491354fb850045203acad176bf82a"}
Jul 06 20:26:17.503883 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:17.503832    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" podStartSLOduration=1.503816762 podStartE2EDuration="1.503816762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:26:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:17.501402101 +0000 UTC m=+2275.763380965" watchObservedRunningTime="2026-07-06 20:26:17.503816762 +0000 UTC m=+2275.765795624"
Jul 06 20:26:21.398548 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:21.398513    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:26:26.919420 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:26.919384    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:26.919781 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:26.919431    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:26.931370 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:26.931345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:26:27.540204 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:26:27.540175    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:27:04.297032 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:04.296993    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:27:18.998303 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:18.998223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:27:27.022312 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:27.022280    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:27:29.428179 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.428144    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:27:29.428636 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.428402    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" podUID="2a4743ce-8514-4ad0-9123-edc7129d0d84" containerName="maas-api" containerID="cri-o://a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b" gracePeriod=30
Jul 06 20:27:29.677795 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.677771    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:27:29.717194 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.717129    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca\") pod \"2a4743ce-8514-4ad0-9123-edc7129d0d84\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") "
Jul 06 20:27:29.717319 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.717240    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") pod \"2a4743ce-8514-4ad0-9123-edc7129d0d84\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") "
Jul 06 20:27:29.717319 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.717309    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q45hr\" (UniqueName: \"kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr\") pod \"2a4743ce-8514-4ad0-9123-edc7129d0d84\" (UID: \"2a4743ce-8514-4ad0-9123-edc7129d0d84\") "
Jul 06 20:27:29.717501 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.717478    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a4743ce-8514-4ad0-9123-edc7129d0d84" (UID: "2a4743ce-8514-4ad0-9123-edc7129d0d84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:27:29.717609 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.717595    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4743ce-8514-4ad0-9123-edc7129d0d84-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:29.719208 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.719189    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a4743ce-8514-4ad0-9123-edc7129d0d84" (UID: "2a4743ce-8514-4ad0-9123-edc7129d0d84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:27:29.719283 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.719266    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr" (OuterVolumeSpecName: "kube-api-access-q45hr") pod "2a4743ce-8514-4ad0-9123-edc7129d0d84" (UID: "2a4743ce-8514-4ad0-9123-edc7129d0d84"). InnerVolumeSpecName "kube-api-access-q45hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:29.818456 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.818433    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q45hr\" (UniqueName: \"kubernetes.io/projected/2a4743ce-8514-4ad0-9123-edc7129d0d84-kube-api-access-q45hr\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:29.818456 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.818455    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a4743ce-8514-4ad0-9123-edc7129d0d84-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:29.859178 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.859155    2565 generic.go:356] "Generic (PLEG): container finished" podID="2a4743ce-8514-4ad0-9123-edc7129d0d84" containerID="a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b" exitCode=0
Jul 06 20:27:29.859270 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.859201    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"
Jul 06 20:27:29.859270 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.859208    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" event={"ID":"2a4743ce-8514-4ad0-9123-edc7129d0d84","Type":"ContainerDied","Data":"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"}
Jul 06 20:27:29.859270 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.859250    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx" event={"ID":"2a4743ce-8514-4ad0-9123-edc7129d0d84","Type":"ContainerDied","Data":"2f8fd70672c86a7cc78c82d647b0d0565eabadc0b8b290e55e13fb9e9de410d2"}
Jul 06 20:27:29.859270 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.859270    2565 scope.go:117] "RemoveContainer" containerID="a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"
Jul 06 20:27:29.867903 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.867885    2565 scope.go:117] "RemoveContainer" containerID="a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"
Jul 06 20:27:29.868199 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:27:29.868180    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b\": container with ID starting with a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b not found: ID does not exist" containerID="a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"
Jul 06 20:27:29.868256 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.868206    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b"} err="failed to get container status \"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b\": rpc error: code = NotFound desc = could not find container \"a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b\": container with ID starting with a5f970a8262d7f608dca5057293ad7c83da3d70bf83f6bf861c0b24f7686be8b not found: ID does not exist"
Jul 06 20:27:29.880639 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.880619    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:27:29.883743 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:29.883726    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c46beb-68755b955d-rtqwx"]
Jul 06 20:27:30.662422 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:30.662391    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a4743ce-8514-4ad0-9123-edc7129d0d84" path="/var/lib/kubelet/pods/2a4743ce-8514-4ad0-9123-edc7129d0d84/volumes"
Jul 06 20:27:33.799210 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:33.799180    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:27:42.041603 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.041541    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" podUID="1f481232-dca4-4e40-b956-f1739b1c911c" containerName="main" containerID="cri-o://6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0" gracePeriod=45
Jul 06 20:27:42.291348 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.291317    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:27:42.325357 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325292    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325402    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325430    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325456    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325496 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325484    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325765 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325537    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4thcp\" (UniqueName: \"kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp\") pod \"1f481232-dca4-4e40-b956-f1739b1c911c\" (UID: \"1f481232-dca4-4e40-b956-f1739b1c911c\") "
Jul 06 20:27:42.325765 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325586    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:42.325765 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325695    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:42.325940 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325782    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home" (OuterVolumeSpecName: "home") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:42.325940 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325828    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-tmp-dir\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.325940 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.325849    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-model-cache\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.327749 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.327721    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp" (OuterVolumeSpecName: "kube-api-access-4thcp") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "kube-api-access-4thcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:42.327900 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.327882    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:27:42.328025 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.328005    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm" (OuterVolumeSpecName: "dshm") pod "1f481232-dca4-4e40-b956-f1739b1c911c" (UID: "1f481232-dca4-4e40-b956-f1739b1c911c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:42.426655 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.426622    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4thcp\" (UniqueName: \"kubernetes.io/projected/1f481232-dca4-4e40-b956-f1739b1c911c-kube-api-access-4thcp\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.426655 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.426652    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f481232-dca4-4e40-b956-f1739b1c911c-tls-certs\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.426824 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.426665    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-dshm\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.426824 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.426676    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f481232-dca4-4e40-b956-f1739b1c911c-home\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:27:42.927389 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.927305    2565 generic.go:356] "Generic (PLEG): container finished" podID="1f481232-dca4-4e40-b956-f1739b1c911c" containerID="6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0" exitCode=0
Jul 06 20:27:42.927389 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.927337    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" event={"ID":"1f481232-dca4-4e40-b956-f1739b1c911c","Type":"ContainerDied","Data":"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"}
Jul 06 20:27:42.927389 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.927360    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"
Jul 06 20:27:42.927600 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.927389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z" event={"ID":"1f481232-dca4-4e40-b956-f1739b1c911c","Type":"ContainerDied","Data":"b89168b93be8c401577040c979fcb130c6a491354fb850045203acad176bf82a"}
Jul 06 20:27:42.927600 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.927406    2565 scope.go:117] "RemoveContainer" containerID="6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"
Jul 06 20:27:42.935897 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.935862    2565 scope.go:117] "RemoveContainer" containerID="6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"
Jul 06 20:27:42.936152 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:27:42.936132    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0\": container with ID starting with 6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0 not found: ID does not exist" containerID="6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"
Jul 06 20:27:42.936196 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.936161    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0"} err="failed to get container status \"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0\": rpc error: code = NotFound desc = could not find container \"6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0\": container with ID starting with 6e06bc563a4bba55d9f59ed06acbeacb2fea79c2cfc23e0d7632d28bc9d4bbc0 not found: ID does not exist"
Jul 06 20:27:42.943850 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.943827    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:27:42.945963 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:42.945940    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a2776d/rate-test-model-a2776d-kserve-6b75fdbcb7-4q26z"]
Jul 06 20:27:44.655141 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:27:44.655112    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f481232-dca4-4e40-b956-f1739b1c911c" path="/var/lib/kubelet/pods/1f481232-dca4-4e40-b956-f1739b1c911c/volumes"
Jul 06 20:28:11.474667 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:11.474633    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:28:11.475173 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:11.474833    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" podUID="d2c245c5-9300-4719-ab9d-d392d59dadef" containerName="maas-api" containerID="cri-o://248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa" gracePeriod=30
Jul 06 20:28:11.919424 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:11.919399    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:28:12.071387 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.071299    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca\") pod \"d2c245c5-9300-4719-ab9d-d392d59dadef\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") "
Jul 06 20:28:12.071387 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.071368    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") pod \"d2c245c5-9300-4719-ab9d-d392d59dadef\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") "
Jul 06 20:28:12.071634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.071437    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64jb9\" (UniqueName: \"kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9\") pod \"d2c245c5-9300-4719-ab9d-d392d59dadef\" (UID: \"d2c245c5-9300-4719-ab9d-d392d59dadef\") "
Jul 06 20:28:12.071682 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.071630    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2c245c5-9300-4719-ab9d-d392d59dadef" (UID: "d2c245c5-9300-4719-ab9d-d392d59dadef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:28:12.073484 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.073453    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9" (OuterVolumeSpecName: "kube-api-access-64jb9") pod "d2c245c5-9300-4719-ab9d-d392d59dadef" (UID: "d2c245c5-9300-4719-ab9d-d392d59dadef"). InnerVolumeSpecName "kube-api-access-64jb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:28:12.073484 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.073471    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2c245c5-9300-4719-ab9d-d392d59dadef" (UID: "d2c245c5-9300-4719-ab9d-d392d59dadef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:28:12.084216 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.084194    2565 generic.go:356] "Generic (PLEG): container finished" podID="d2c245c5-9300-4719-ab9d-d392d59dadef" containerID="248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa" exitCode=0
Jul 06 20:28:12.084336 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.084259    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"
Jul 06 20:28:12.084336 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.084276    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" event={"ID":"d2c245c5-9300-4719-ab9d-d392d59dadef","Type":"ContainerDied","Data":"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"}
Jul 06 20:28:12.084464 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.084341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm" event={"ID":"d2c245c5-9300-4719-ab9d-d392d59dadef","Type":"ContainerDied","Data":"e455bdc5e03930a44253313d82a95bf04ea2d26befef92e5cfe861a22db67d00"}
Jul 06 20:28:12.084464 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.084372    2565 scope.go:117] "RemoveContainer" containerID="248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"
Jul 06 20:28:12.093010 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.092983    2565 scope.go:117] "RemoveContainer" containerID="248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"
Jul 06 20:28:12.093252 ip-10-0-132-142 kubenswrapper[2565]: E0706 20:28:12.093232    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa\": container with ID starting with 248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa not found: ID does not exist" containerID="248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"
Jul 06 20:28:12.093325 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.093259    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa"} err="failed to get container status \"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa\": rpc error: code = NotFound desc = could not find container \"248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa\": container with ID starting with 248f04b8e20b88a38ed705e89b445ba859be3f3f2351bfb468e63fb15a1d55aa not found: ID does not exist"
Jul 06 20:28:12.105936 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.105901    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:28:12.111382 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.111364    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a2776d-d5644444b-lv4nm"]
Jul 06 20:28:12.172050 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.172028    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2c245c5-9300-4719-ab9d-d392d59dadef-maas-api-tls\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:28:12.172050 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.172051    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64jb9\" (UniqueName: \"kubernetes.io/projected/d2c245c5-9300-4719-ab9d-d392d59dadef-kube-api-access-64jb9\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:28:12.172190 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.172060    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c245c5-9300-4719-ab9d-d392d59dadef-openshift-service-ca\") on node \"ip-10-0-132-142.ec2.internal\" DevicePath \"\""
Jul 06 20:28:12.655994 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:12.655879    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c245c5-9300-4719-ab9d-d392d59dadef" path="/var/lib/kubelet/pods/d2c245c5-9300-4719-ab9d-d392d59dadef/volumes"
Jul 06 20:28:13.998364 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:13.998328    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mts8f"]
Jul 06 20:28:18.525760 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:18.525724    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b84b49d-7qgnt_026f78c3-544f-4f9f-97bd-5d092ef1c9c9/authorino/0.log"
Jul 06 20:28:22.804296 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:22.804269    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:28:22.811638 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:22.811618    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58fr7_3395b714-6786-482e-8ee4-75e554b601be/ovn-acl-logging/0.log"
Jul 06 20:28:23.983880 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:23.983850    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722815-fzvwd_4892111f-1fb0-45a1-b2dc-401bdbfafe2d/cleanup/0.log"
Jul 06 20:28:24.243993 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:24.243896    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vvf4r_2334aa6f-f988-4811-842b-346ead0e42c3/server/0.log"
Jul 06 20:28:24.371291 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:24.371265    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nddfk_c09eefa8-7f7a-4733-bfd2-2b2d22c3225b/manager/0.log"
Jul 06 20:28:24.752118 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:24.752093    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-mlztn_4e97320e-57bb-4143-bca7-1f180b3166e0/copy-manifests/0.log"
Jul 06 20:28:24.759144 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:24.759118    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-mlztn_4e97320e-57bb-4143-bca7-1f180b3166e0/manager/0.log"
Jul 06 20:28:25.713683 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.713653    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/util/0.log"
Jul 06 20:28:25.720811 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.720787    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/pull/0.log"
Jul 06 20:28:25.727724 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.727700    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/extract/0.log"
Jul 06 20:28:25.843725 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.843705    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/util/0.log"
Jul 06 20:28:25.850591 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.850570    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/pull/0.log"
Jul 06 20:28:25.857264 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.857246    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/extract/0.log"
Jul 06 20:28:25.977982 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.977905    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/extract/0.log"
Jul 06 20:28:25.984799 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.984781    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/util/0.log"
Jul 06 20:28:25.991786 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:25.991764    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/pull/0.log"
Jul 06 20:28:26.147353 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.147328    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/util/0.log"
Jul 06 20:28:26.173584 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.173533    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/pull/0.log"
Jul 06 20:28:26.183884 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.183857    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/extract/0.log"
Jul 06 20:28:26.328701 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.328679    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b84b49d-7qgnt_026f78c3-544f-4f9f-97bd-5d092ef1c9c9/authorino/0.log"
Jul 06 20:28:26.458066 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.458038    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-slw9h_484940eb-bdc7-4d96-a8a4-3b449b1d7672/manager/0.log"
Jul 06 20:28:26.588003 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.587942    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6khmk_961fa3fe-5cfd-4f70-977d-9c3c6baef362/manager/0.log"
Jul 06 20:28:26.715673 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.715652    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-bn2fn_ff4a89fd-6a0f-4abb-8721-fb90346e7e76/kuadrant-console-plugin/0.log"
Jul 06 20:28:26.849824 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:26.849756    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ll29l_4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0/registry-server/0.log"
Jul 06 20:28:27.098835 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:27.098807    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mts8f_5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef/limitador/0.log"
Jul 06 20:28:27.231108 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:27.231037    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jb9j5_15a41293-96b5-4785-bf37-df3800d321f2/manager/0.log"
Jul 06 20:28:27.822478 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:27.822448    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-747cfbfd67-x2zzn_26160051-315c-4b17-80e5-013261520b9d/kube-auth-proxy/0.log"
Jul 06 20:28:28.928335 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:28.928308    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-fsbff_085ee9ab-bac0-4646-86cb-3eba8fc3e5c8/main/0.log"
Jul 06 20:28:29.053144 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:29.053119    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xvbkv_ffb21700-4982-471f-82c5-f5e7a7803d3e/main/0.log"
Jul 06 20:28:29.287965 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:29.287936    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2kkp8_d1510425-b50b-4963-9b1c-f353bbc9dc00/main/0.log"
Jul 06 20:28:29.402866 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:29.402834    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-n6bvv_a1e7abcd-7cb1-453f-a61a-39bafb6bcb85/main/0.log"
Jul 06 20:28:33.976069 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.976028    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4b4qp/must-gather-zvcpx"]
Jul 06 20:28:33.977315 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977286    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a4743ce-8514-4ad0-9123-edc7129d0d84" containerName="maas-api"
Jul 06 20:28:33.977517 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977498    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a4743ce-8514-4ad0-9123-edc7129d0d84" containerName="maas-api"
Jul 06 20:28:33.977581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977530    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f481232-dca4-4e40-b956-f1739b1c911c" containerName="main"
Jul 06 20:28:33.977581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977540    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f481232-dca4-4e40-b956-f1739b1c911c" containerName="main"
Jul 06 20:28:33.977581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977569    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c245c5-9300-4719-ab9d-d392d59dadef" containerName="maas-api"
Jul 06 20:28:33.977581 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977577    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c245c5-9300-4719-ab9d-d392d59dadef" containerName="maas-api"
Jul 06 20:28:33.977796 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977778    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a4743ce-8514-4ad0-9123-edc7129d0d84" containerName="maas-api"
Jul 06 20:28:33.977854 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977808    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f481232-dca4-4e40-b956-f1739b1c911c" containerName="main"
Jul 06 20:28:33.977854 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.977819    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c245c5-9300-4719-ab9d-d392d59dadef" containerName="maas-api"
Jul 06 20:28:33.981677 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.981642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:33.984572 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.984535    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b4qp\"/\"kube-root-ca.crt\""
Jul 06 20:28:33.985365 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.985338    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b4qp\"/\"openshift-service-ca.crt\""
Jul 06 20:28:33.985513 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.985401    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4b4qp\"/\"default-dockercfg-djtng\""
Jul 06 20:28:33.987033 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:33.987011    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/must-gather-zvcpx"]
Jul 06 20:28:34.047478 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.047453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4xf\" (UniqueName: \"kubernetes.io/projected/f255f41b-d91f-4ea6-a053-23799bee9d2e-kube-api-access-7v4xf\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.047570 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.047488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f255f41b-d91f-4ea6-a053-23799bee9d2e-must-gather-output\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.148759 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.148733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f255f41b-d91f-4ea6-a053-23799bee9d2e-must-gather-output\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.148858 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.148796    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4xf\" (UniqueName: \"kubernetes.io/projected/f255f41b-d91f-4ea6-a053-23799bee9d2e-kube-api-access-7v4xf\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.149060 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.149041    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f255f41b-d91f-4ea6-a053-23799bee9d2e-must-gather-output\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.156606 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.156588    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4xf\" (UniqueName: \"kubernetes.io/projected/f255f41b-d91f-4ea6-a053-23799bee9d2e-kube-api-access-7v4xf\") pod \"must-gather-zvcpx\" (UID: \"f255f41b-d91f-4ea6-a053-23799bee9d2e\") " pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.291864 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.291834    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/must-gather-zvcpx"
Jul 06 20:28:34.438178 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.438143    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/must-gather-zvcpx"]
Jul 06 20:28:34.441208 ip-10-0-132-142 kubenswrapper[2565]: W0706 20:28:34.441171    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf255f41b_d91f_4ea6_a053_23799bee9d2e.slice/crio-d6741abc0fbea6506d7051521d628e2dd3c42f2352050bc35abcced42b6aa730 WatchSource:0}: Error finding container d6741abc0fbea6506d7051521d628e2dd3c42f2352050bc35abcced42b6aa730: Status 404 returned error can't find the container with id d6741abc0fbea6506d7051521d628e2dd3c42f2352050bc35abcced42b6aa730
Jul 06 20:28:34.443124 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:34.443104    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:28:35.202474 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:35.202442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/must-gather-zvcpx" event={"ID":"f255f41b-d91f-4ea6-a053-23799bee9d2e","Type":"ContainerStarted","Data":"d6741abc0fbea6506d7051521d628e2dd3c42f2352050bc35abcced42b6aa730"}
Jul 06 20:28:36.213459 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.213425    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/must-gather-zvcpx" event={"ID":"f255f41b-d91f-4ea6-a053-23799bee9d2e","Type":"ContainerStarted","Data":"e1ba2a29d30e3edd072e00530c67833efe75cf2f1696a01a8737e93e82dd9a59"}
Jul 06 20:28:36.213965 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.213472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/must-gather-zvcpx" event={"ID":"f255f41b-d91f-4ea6-a053-23799bee9d2e","Type":"ContainerStarted","Data":"e440d386ddac12dfad19d8aa30330190eb3f6870af56a4cad0a9638533703578"}
Jul 06 20:28:36.233857 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.233811    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4b4qp/must-gather-zvcpx" podStartSLOduration=2.5250840869999998 podStartE2EDuration="3.233795578s" totalImagesPullingTime="708.711491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:28:36.232097033 +0000 UTC m=+2414.494075896" watchObservedRunningTime="2026-07-06 20:28:36.233795578 +0000 UTC m=+2414.495774480"
Jul 06 20:28:36.695470 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.695442    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sh2cx_1605fa02-16bb-41b4-82fb-c934c9c21dc4/global-pull-secret-syncer/0.log"
Jul 06 20:28:36.784577 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.784543    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qsd29_653fa34b-0f62-40a7-86f7-c36abf2cdccb/konnectivity-agent/0.log"
Jul 06 20:28:36.861140 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:36.861113    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-142.ec2.internal_195224f1370a48f9ab6e557f0d094f7d/haproxy/0.log"
Jul 06 20:28:40.900755 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:40.900729    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/extract/0.log"
Jul 06 20:28:40.954929 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:40.953269    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/util/0.log"
Jul 06 20:28:41.001122 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.001091    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc4rq_64231890-1a99-4aa1-8c56-5325e0261d05/pull/0.log"
Jul 06 20:28:41.059769 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.059739    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/extract/0.log"
Jul 06 20:28:41.099948 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.098756    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/util/0.log"
Jul 06 20:28:41.152208 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.152134    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vtxvm_831f918f-f685-4667-b76e-1ea3f5af7a91/pull/0.log"
Jul 06 20:28:41.200163 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.200124    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/extract/0.log"
Jul 06 20:28:41.247564 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.247537    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/util/0.log"
Jul 06 20:28:41.292053 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.292021    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qtvxk_325616cd-18f5-448e-bd27-75f394567b90/pull/0.log"
Jul 06 20:28:41.342096 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.342063    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/extract/0.log"
Jul 06 20:28:41.387904 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.387873    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/util/0.log"
Jul 06 20:28:41.434255 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.434158    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fkrxn_0a0cd019-292c-4c35-8069-318eeea30798/pull/0.log"
Jul 06 20:28:41.877746 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.877364    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b84b49d-7qgnt_026f78c3-544f-4f9f-97bd-5d092ef1c9c9/authorino/0.log"
Jul 06 20:28:41.914064 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.914033    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-slw9h_484940eb-bdc7-4d96-a8a4-3b449b1d7672/manager/0.log"
Jul 06 20:28:41.954206 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.954180    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6khmk_961fa3fe-5cfd-4f70-977d-9c3c6baef362/manager/0.log"
Jul 06 20:28:41.996693 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:41.996664    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-bn2fn_ff4a89fd-6a0f-4abb-8721-fb90346e7e76/kuadrant-console-plugin/0.log"
Jul 06 20:28:42.051732 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:42.051695    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ll29l_4d2b2f55-7b56-44a6-86c7-00a82bb8c8f0/registry-server/0.log"
Jul 06 20:28:42.200082 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:42.199999    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mts8f_5f61059a-00b0-4cb9-a2b3-67eb1f6b35ef/limitador/0.log"
Jul 06 20:28:42.297948 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:42.297897    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jb9j5_15a41293-96b5-4785-bf37-df3800d321f2/manager/0.log"
Jul 06 20:28:44.074837 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:44.074730    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-m7hc7_d4863925-0fbb-4fdc-9ca8-0a8f4dda91d0/cluster-monitoring-operator/0.log"
Jul 06 20:28:44.315235 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:44.315199    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97p8z_c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4/node-exporter/0.log"
Jul 06 20:28:44.342227 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:44.342156    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97p8z_c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4/kube-rbac-proxy/0.log"
Jul 06 20:28:44.366907 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:44.366878    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97p8z_c939f3d9-44f1-432c-bc4e-f4e9bd51ddb4/init-textfile/0.log"
Jul 06 20:28:46.163132 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.163099    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-sg4dn_4705762e-ae4b-48d5-8fa1-a25b47b62d82/networking-console-plugin/0.log"
Jul 06 20:28:46.319242 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.319152    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"]
Jul 06 20:28:46.323254 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.323229    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.346133 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.346103    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"]
Jul 06 20:28:46.380174 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.380134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-proc\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.380358 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.380216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mwp4\" (UniqueName: \"kubernetes.io/projected/19c29e81-ff10-43a1-b955-3bd83f360e3b-kube-api-access-5mwp4\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.380358 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.380239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-podres\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.380358 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.380270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-sys\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.380358 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.380298    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-lib-modules\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.481438 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481399    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwp4\" (UniqueName: \"kubernetes.io/projected/19c29e81-ff10-43a1-b955-3bd83f360e3b-kube-api-access-5mwp4\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.481691 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-podres\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.481848 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-sys\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.481991 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-lib-modules\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.482127 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.482102    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-lib-modules\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.482309 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481866    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-podres\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.482396 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.482172    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-proc\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.482570 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.482228    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-proc\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.482634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.481926    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c29e81-ff10-43a1-b955-3bd83f360e3b-sys\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.490634 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.490608    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mwp4\" (UniqueName: \"kubernetes.io/projected/19c29e81-ff10-43a1-b955-3bd83f360e3b-kube-api-access-5mwp4\") pod \"perf-node-gather-daemonset-jzf9r\" (UID: \"19c29e81-ff10-43a1-b955-3bd83f360e3b\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.635510 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.635427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:46.832265 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:46.832236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"]
Jul 06 20:28:47.281936 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.281891    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-558b86594d-m6h5m_9e752371-bad5-4189-b549-af098ff8c2aa/console/0.log"
Jul 06 20:28:47.300224 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.300190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r" event={"ID":"19c29e81-ff10-43a1-b955-3bd83f360e3b","Type":"ContainerStarted","Data":"ecc1fb9beaa5705111966f7bbaef5a37724ddd3a0ab9b66d380a1b2529bada63"}
Jul 06 20:28:47.300395 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.300253    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r" event={"ID":"19c29e81-ff10-43a1-b955-3bd83f360e3b","Type":"ContainerStarted","Data":"1d4d007ddcceda5671061a4678804f83b1fd94e9832de96b8406e66f10bc0fe6"}
Jul 06 20:28:47.300457 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.300395    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:47.320410 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.320362    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r" podStartSLOduration=1.3203475120000001 podStartE2EDuration="1.320347512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:28:47.319261405 +0000 UTC m=+2425.581240268" watchObservedRunningTime="2026-07-06 20:28:47.320347512 +0000 UTC m=+2425.582326374"
Jul 06 20:28:47.859354 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:47.859326    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-jjvr6_7c943477-a00f-49fd-b404-20b64c51702b/volume-data-source-validator/0.log"
Jul 06 20:28:48.821564 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:48.821531    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-24ntk_52abd723-4417-452c-b72a-b3f5800f2762/dns/0.log"
Jul 06 20:28:48.843562 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:48.843533    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-24ntk_52abd723-4417-452c-b72a-b3f5800f2762/kube-rbac-proxy/0.log"
Jul 06 20:28:48.969342 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:48.969313    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42sf8_4a928635-994c-416e-aba8-35d28625546d/dns-node-resolver/0.log"
Jul 06 20:28:49.558528 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:49.558486    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-922hx_31a51b30-fae6-46a6-8202-a561d20de2e9/node-ca/0.log"
Jul 06 20:28:50.630169 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:50.630142    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-747cfbfd67-x2zzn_26160051-315c-4b17-80e5-013261520b9d/kube-auth-proxy/0.log"
Jul 06 20:28:51.528975 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:51.528947    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tsgnw_9731f41f-d036-4b90-a4ec-b6963f26c4cb/serve-healthcheck-canary/0.log"
Jul 06 20:28:52.002137 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:52.002052    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z8jt_1a6bfb02-7938-4c5f-a249-cfcd5b428ad4/kube-rbac-proxy/0.log"
Jul 06 20:28:52.024064 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:52.024038    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z8jt_1a6bfb02-7938-4c5f-a249-cfcd5b428ad4/exporter/0.log"
Jul 06 20:28:52.048498 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:52.048470    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z8jt_1a6bfb02-7938-4c5f-a249-cfcd5b428ad4/extractor/0.log"
Jul 06 20:28:53.318486 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:53.318447    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-jzf9r"
Jul 06 20:28:54.532070 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:54.532038    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722815-fzvwd_4892111f-1fb0-45a1-b2dc-401bdbfafe2d/cleanup/0.log"
Jul 06 20:28:54.656497 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:54.656472    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vvf4r_2334aa6f-f988-4811-842b-346ead0e42c3/server/0.log"
Jul 06 20:28:54.760294 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:54.759641    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nddfk_c09eefa8-7f7a-4733-bfd2-2b2d22c3225b/manager/0.log"
Jul 06 20:28:54.938852 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:54.938824    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-mlztn_4e97320e-57bb-4143-bca7-1f180b3166e0/manager/0.log"
Jul 06 20:28:54.961756 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:54.961728    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-mlztn_4e97320e-57bb-4143-bca7-1f180b3166e0/copy-manifests/0.log"
Jul 06 20:28:56.632295 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:56.632219    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-747cfbfd67-x2zzn_26160051-315c-4b17-80e5-013261520b9d/kube-auth-proxy/0.log"
Jul 06 20:28:58.071706 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:28:58.071668    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-747cfbfd67-x2zzn_26160051-315c-4b17-80e5-013261520b9d/kube-auth-proxy/0.log"
Jul 06 20:29:01.501654 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:01.501618    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hjc86_a34011c5-eb88-4ca9-9b3f-26b0235ccc09/kube-storage-version-migrator-operator/1.log"
Jul 06 20:29:01.503350 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:01.503326    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hjc86_a34011c5-eb88-4ca9-9b3f-26b0235ccc09/kube-storage-version-migrator-operator/0.log"
Jul 06 20:29:02.811044 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.811016    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/kube-multus-additional-cni-plugins/0.log"
Jul 06 20:29:02.833075 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.833046    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/egress-router-binary-copy/0.log"
Jul 06 20:29:02.856179 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.856156    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/cni-plugins/0.log"
Jul 06 20:29:02.877598 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.877577    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/bond-cni-plugin/0.log"
Jul 06 20:29:02.898867 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.898838    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/routeoverride-cni/0.log"
Jul 06 20:29:02.921262 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.921195    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/whereabouts-cni-bincopy/0.log"
Jul 06 20:29:02.946324 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:02.946295    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkn8j_3b2a4ba1-2f43-45b8-9d5f-443c16c85870/whereabouts-cni/0.log"
Jul 06 20:29:03.029419 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:03.029384    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n9nkg_94ef51cc-2713-49f4-bc37-d199538fa6cc/kube-multus/0.log"
Jul 06 20:29:03.059235 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:03.059195    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pzghn_dc51274b-629e-42ac-a025-e9ffafd363f5/network-metrics-daemon/0.log"
Jul 06 20:29:03.083949 ip-10-0-132-142 kubenswrapper[2565]: I0706 20:29:03.083907    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pzghn_dc51274b-629e-42ac-a025-e9ffafd363f5/kube-rbac-proxy/0.log"